[Cod-bugs] Query about terms of use

[Andrius Merkys]

Dear Mark,

On 2024-04-04 12:25, SPILLMAN Mark wrote:
> I am writing to ask you if there is any guidance on acceptable “terms of 
> use” for querying the OPTIMADE endpoint, for example, guidance on the 
> number of queries a user should submit within a given time, or any other 
> limits on use (e.g. rate limits, concurrent requests etc).
> 
> I tried to find something to that effect on the website and also via the 
> /info endpoint, but wasn’t able to see anything.

You are right, at the moment there are no explicit terms of use 
regarding the COD server load. In the upcoming OPTIMADE release v1.2.0 
there is going to be a field 'request_delay' providing the back-off time 
in seconds before the next request [1].

I think the sensible amount of time between requests from the same 
source should be at least 0.5s in order not to overload our servers. 
Concurrent requests are allowed, but the same guidelines should be 
respected. This means that posting concurrent requests on two parallel 
treads every second should be fine.

At the moment the COD does not enforce these guidelines. Nevertheless we 
would like to kindly ask the COD users to introduce sensible delays in 
their software if it is going to make many requests to the COD.

> I also wanted to ask if there was an obvious way to find the maximum 
> page_limit argument for OPTIMADE requests – it seems to vary depending 
> on which parameters are requested. For example, if I include 
> &response_fields=cartesian_site_positions in my query, then page_limit 
> max is 10. However, if this is not included, then page_limit max is 100.

There is no obvious way to find the maximum limits at the moment. The 
COD uses 'page_limit' of 100 for queries not needing atom site 
information, and 10 for queries which do. The limit is made to be lower 
for resource-intensive queries as site information in the COD is 
extracted on-the-fly at the moment. Once we make this more effective, 
the maximum limit on 'page_limit' will be increased.

[1] https://github.com/Materials-Consortia/OPTIMADE/pull/411

Best regards,
Andrius

-- 
Andrius Merkys
Vilnius University Institute of Biotechnology, Saulėtekio al. 7
LT-10257 Vilnius, Lithuania

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.