There are a number of ways to proceed with open rank searches. The easiest is to clearly identify how to apply as a junior and a senior candidate in the "application instructions" section of your landing page, including specific instructions that junior rank candidates should submit three letters of reference and senior candidates should submit a list of five referees.
When you choose your required documents, choose "confidential letter of reference," but set the required number to zero, then click "allow applicants to upload additional material." This will allow junior candidates to upload their three letters without requiring any letters of senior candidates.
For senior rank candidates, follow the process detailed above under "I am running a senior-rank search," but set the required number of other documents (references) to zero.