From mehmet.giritli at stfc.ac.uk Mon Nov 17 16:08:23 2025 From: mehmet.giritli at stfc.ac.uk (Mehmet Giritli - STFC UKRI) Date: Mon, 17 Nov 2025 14:08:23 +0000 Subject: [Cod-bugs] Authors matching in Optimade API Message-ID: Dear friends, I am trying to use Optimade API to search using author names. But I noticed a behaviour which is a bit unexpected. Consider this query: authors.name HAS "A. Day" The spec for HAS operator says that this should only match if there is an exact match, i.e., there is an author which is equal to "A. Day", in the record. Here is COD's output: https://www.crystallography.net/cod/optimade/references?filter=authors.name%20HAS%20%22A.%20Day%22 The single record above includes an author "A. Dayalan". However, I think this should not be the case. It seems like "HAS" implementation behaves like what I would expect from "CONTAINS" implementation. But, this means that there is no way to get only the records which contain values we are explicit about. That said, I also noticed that CONTAINS is not implemented for author.names at the moment: https://www.crystallography.net/cod/optimade/references?filter=authors.name%20CONTAINS%20%22A.%20Day%22 To sum up my request, do you think it is possible to: 1. Change implementation of "author.names HAS" so that it allows only exact matching. 2. Implement "author.names CONTAINS" which is basically the current implementation you have for HAS. I hope it made sense. Please let me know if there is a way to overcome this situation currently (e.g., to exact match an author) in your API. Kind Regards, Mehmet. -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrius.merkys at gmail.com Mon Nov 17 16:21:38 2025 From: andrius.merkys at gmail.com (Andrius Merkys) Date: Mon, 17 Nov 2025 16:21:38 +0200 Subject: [Cod-bugs] Authors matching in Optimade API In-Reply-To: References: Message-ID: <2435406c-7923-4ba3-9b54-315fce90f93a@gmail.com> Dear Mehmet, On 2025-11-17 16:08, Mehmet Giritli - STFC UKRI wrote: > I am trying to use Optimade API to search using author names. But I > noticed a behaviour which is a bit unexpected. > > Consider this query: > > authors.name HAS "A. Day" > > The spec for HAS operator says that this should only match if there is > an exact match, i.e., there is an author which is equal to "A. Day", in > the record. > > Here is COD's output: > https://www.crystallography.net/cod/optimade/references?filter=authors.name%20HAS%20%22A.%20Day%22 > > The single record above includes an author "A. Dayalan". However, I > think this should not be the case. ?It seems like "HAS" implementation > behaves like what I would expect from "CONTAINS" implementation. But, > this means that there is no way to get only the records which contain > values we are explicit about. You are right, here the COD's OPTIMADE implementation deviates slightly from the OPTIMADE specification. Given our backend, implementing support for such queries to the letter has proven to be too complicated, thus at the moment we have this deviation. I think it would be nice to at least emit a warning from our side. > That said, I also noticed that CONTAINS is not implemented for > author.names at the moment: > > https://www.crystallography.net/cod/optimade/references?filter=authors.name%20CONTAINS%20%22A.%20Day%22 In this case the filter string should be: authors.name HAS CONTAINS "A. Day" (Notice the addition of 'HAS'). 'CONTAINS' alone is an operator acting on strings, not on lists. > To sum up my request, do you think it is possible to: > > 1. > Change implementation of "author.names HAS" so that it allows only > exact matching. This should be doable, it just needs additional work from our side. > 2. > Implement "author.names CONTAINS" which is basically the current > implementation you have for HAS. As explained, 'author.names CONTAINS' should be changed to 'author.names HAS CONTAINS', as 'author.names CONTAINS' should not work on any OPTIMADE implementation. > I hope it made sense. Please let me know if there is a way to overcome > this situation currently (e.g., to exact match an author) in your API. Thanks for contacting us regarding this issue. Best wishes, Andrius -- Andrius Merkys Vilnius University Institute of Biotechnology, Saul?tekio al. 7 LT-10257 Vilnius, Lithuania -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From mehmet.giritli at stfc.ac.uk Mon Nov 17 17:03:39 2025 From: mehmet.giritli at stfc.ac.uk (Mehmet Giritli - STFC UKRI) Date: Mon, 17 Nov 2025 15:03:39 +0000 Subject: [Cod-bugs] Authors matching in Optimade API In-Reply-To: <2435406c-7923-4ba3-9b54-315fce90f93a@gmail.com> References: <2435406c-7923-4ba3-9b54-315fce90f93a@gmail.com> Message-ID: Dear Andrius, Thank you so much for your quick response. We are looking forward to have exact matches in COD. Thank you so much! Regarding "HAS CONTAINS", this does not seem to work with the Lark validator we have. In our system, we use a query validator to check for valid optimade filters. This is done by the Lark language interpreter which the optimade library uses. It gives an error when this combination is used. Here is an example: Unable to parse filter ( _psdi_authors HAS CONTAINS " Day " ). Lark traceback: No terminal matches 'C' in the current parser context, at line 1 col 21 ( _psdi_authors HAS CONTAINS " Day " ) ^ Expected one of: * SIGNED_INT * OPERATOR * FALSE * SIGNED_FLOAT * ESCAPED_STRING * TRUE * ALL * IDENTIFIER * ANY * ONLY May I ask you, whenever you get around to it, to double check with Optimade spec if this is allowed please? Unless it is a legal Optimade query, we will not be able to use it. Moreover, Matthew Evans has done a private API for us and CONTAINS operator in the way I showed above is how we use it (e.g., author.names CONTAINS "..."). I might be wrong, but may be the "authors" is the list here and "authors.name" is actually a string? I think Mattew Evans is the best person to answer that, I haven't looked to the Optimade spec for a while now ? Once again, thank you for your quick response. I'll keep on checking to see if exact matches are available for us to use ? Best, Mehmet. ________________________________ From: Andrius Merkys Sent: 17 November 2025 14:21 To: Giritli, Mehmet (STFC,RAL,SC) Cc: cod-bugs at ibt.lt Subject: Re: [Cod-bugs] Authors matching in Optimade API Dear Mehmet, On 2025-11-17 16:08, Mehmet Giritli - STFC UKRI wrote: > I am trying to use Optimade API to search using author names. But I > noticed a behaviour which is a bit unexpected. > > Consider this query: > > authors.name HAS "A. Day" > > The spec for HAS operator says that this should only match if there is > an exact match, i.e., there is an author which is equal to "A. Day", in > the record. > > Here is COD's output: > https://www.crystallography.net/cod/optimade/references?filter=authors.name%20HAS%20%22A.%20Day%22 > > The single record above includes an author "A. Dayalan". However, I > think this should not be the case. It seems like "HAS" implementation > behaves like what I would expect from "CONTAINS" implementation. But, > this means that there is no way to get only the records which contain > values we are explicit about. You are right, here the COD's OPTIMADE implementation deviates slightly from the OPTIMADE specification. Given our backend, implementing support for such queries to the letter has proven to be too complicated, thus at the moment we have this deviation. I think it would be nice to at least emit a warning from our side. > That said, I also noticed that CONTAINS is not implemented for > author.names at the moment: > > https://www.crystallography.net/cod/optimade/references?filter=authors.name%20CONTAINS%20%22A.%20Day%22 In this case the filter string should be: authors.name HAS CONTAINS "A. Day" (Notice the addition of 'HAS'). 'CONTAINS' alone is an operator acting on strings, not on lists. > To sum up my request, do you think it is possible to: > > 1. > Change implementation of "author.names HAS" so that it allows only > exact matching. This should be doable, it just needs additional work from our side. > 2. > Implement "author.names CONTAINS" which is basically the current > implementation you have for HAS. As explained, 'author.names CONTAINS' should be changed to 'author.names HAS CONTAINS', as 'author.names CONTAINS' should not work on any OPTIMADE implementation. > I hope it made sense. Please let me know if there is a way to overcome > this situation currently (e.g., to exact match an author) in your API. Thanks for contacting us regarding this issue. Best wishes, Andrius -- Andrius Merkys Vilnius University Institute of Biotechnology, Saul?tekio al. 7 LT-10257 Vilnius, Lithuania -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrius.merkys at gmail.com Tue Nov 18 09:49:40 2025 From: andrius.merkys at gmail.com (Andrius Merkys) Date: Tue, 18 Nov 2025 09:49:40 +0200 Subject: [Cod-bugs] Authors matching in Optimade API In-Reply-To: References: <2435406c-7923-4ba3-9b54-315fce90f93a@gmail.com> Message-ID: <4c3a99bb-a73f-4991-8f72-4de6a9abad8a@gmail.com> Dear Mehmet, On 2025-11-17 17:03, Mehmet Giritli - STFC UKRI wrote: > We are looking forward to have exact matches in COD. Thank you so much! Great - we will look into implementing these queries to the letter of the specification. > Regarding "HAS CONTAINS", this does not seem to work with the Lark > validator we have. In our system, we use a query validator to check for > valid optimade filters. This is done by the Lark language interpreter > which the optimade library uses. It gives an error when this combination > is used. Here is an example: > > > Unable to parse filter ( _psdi_authors HAS CONTAINS " Day " ). Lark > traceback: > No terminal matches 'C' in the current parser context, at line 1 col 21 > > ( _psdi_authors HAS CONTAINS " Day " ) > ? ? ? ? ? ? ? ? ? ? ^ > Expected one of: > ? ? ? ? * SIGNED_INT > ? ? ? ? * OPERATOR > ? ? ? ? * FALSE > ? ? ? ? * SIGNED_FLOAT > ? ? ? ? * ESCAPED_STRING > ? ? ? ? * TRUE > ? ? ? ? * ALL > ? ? ? ? * IDENTIFIER > ? ? ? ? * ANY > ? ? ? ? * ONLY > > May I ask you, whenever you get around to it, to double check with > Optimade spec if this is allowed please? Unless it is a legal Optimade > query, we will not be able to use it. Moreover, Matthew Evans has done a > private API for us and CONTAINS operator in the way I showed above is > how we use it (e.g., author.names CONTAINS "..."). I might be wrong, but > may be the "authors" is the list here and "authors.name" is actually a > string? I think Mattew Evans is the best person to answer that, I > haven't looked to the Optimade spec for a while now ? Queries like 'HAS CONTAINS' were only made possible since the OPTIMADE specification release v1.2.0 last year (https://github.com/Materials-Consortia/OPTIMADE/pull/415). Prior to v1.2.0 they were not supported. I have a hunch that Lark grammar has not been updated since v1.2.0. We may pull Matthew to the discussion to check the status of the grammar. Best wishes, Andrius -- Andrius Merkys Vilnius University Institute of Biotechnology, Saul?tekio al. 7 LT-10257 Vilnius, Lithuania -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From tani.yosuke.y1 at f.mail.nagoya-u.ac.jp Fri Nov 21 04:09:50 2025 From: tani.yosuke.y1 at f.mail.nagoya-u.ac.jp (TANI Yosuke) Date: Fri, 21 Nov 2025 02:09:50 +0000 Subject: [Cod-bugs] trouble in sign-up Message-ID: Dear COD supports, I tried to sign up my account from following page, but I haven't receive an email to confirm. https://www.crystallography.net/cod/check_user.php?signup I used this email address (tani.yosuke.y1 at f.mail.nagoya-u.ac.jp), and checked the spam folder but could not find a confirmation email. When I tried again to sign up with the same information (username and email address), following message appeared: "Username 'y-tani' is already taken -- please choose a different username." When I tried "forgot passward" option, then: "User with given username 'y-tani' and email 'tani.yosuke.y1 at f.mail.nagoya-u.ac.jp' does not exist" Also, I tried with a different address (gmail), but again it didn't work. Could you please help solving this issue? Thank you for your kind support. Best regards, Yosuke ????????????????????????? ?? ???Yosuke Tani, PhD ???????????????? ??????????????????????????ITbM? ??????????????? ?????? ?Designated Associate Professor ?Institute of Transformative Bio-Molecules (ITbM) ?Nagoya University ? ?tel: +81-52-789-5248 ?email: tani.yosuke.y1 at f.mail.nagoya-u.ac.jp ????????????????????????? -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrius.merkys at gmail.com Fri Nov 21 10:03:01 2025 From: andrius.merkys at gmail.com (Andrius Merkys) Date: Fri, 21 Nov 2025 10:03:01 +0200 Subject: [Cod-bugs] trouble in sign-up In-Reply-To: References: Message-ID: <02e777fb-7519-45e1-ae2f-e8a01ea74282@gmail.com> Dear Yosuke, On 2025-11-21 04:09, TANI Yosuke wrote: > I tried to sign up my account from following page, but I haven't receive > an email to confirm. > https://www.crystallography.net/cod/check_user.php?signup > In case your confirmation email did not arrive, here is the link for your account activation: https://www.crystallography.net/cod/check_user.php?activationstr=b4581f488e82a3138d395eadc1d17626 Clicking on this link should activate your account. Please let us know should you run into any additional obstacles. Best wishes, Andrius -- Andrius Merkys Vilnius University Institute of Biotechnology, Saul?tekio al. 7 LT-10257 Vilnius, Lithuania -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From tani.yosuke.y1 at f.mail.nagoya-u.ac.jp Fri Nov 21 11:23:41 2025 From: tani.yosuke.y1 at f.mail.nagoya-u.ac.jp (TANI Yosuke) Date: Fri, 21 Nov 2025 09:23:41 +0000 Subject: [Cod-bugs] trouble in sign-up In-Reply-To: <02e777fb-7519-45e1-ae2f-e8a01ea74282@gmail.com> References: <02e777fb-7519-45e1-ae2f-e8a01ea74282@gmail.com> Message-ID: Dear Andrius, Thank you very much for your prompt reply. I have successfully logged in my account. I've just deposited a structure #3000627, whose data was collected by MicroED. We refined the atomic model only kinematically, ignoring effects of multiple scattering (dynamical diffraction). This led to high R1 and wR2 values. To deposite this data, I deleted following lines from the cif file: _refine_ls_R_factor_gt 0.1704 _refine_ls_wR_factor_ref 0.4086 Could you please add these lines to the structure #3000627? Thank you for your kind assistance. Best wishes, Yosuke ________________________________ ???: Andrius Merkys ????: 2025?11?21? 17:03 ??: TANI Yosuke CC: cod-bugs at ibt.lt ??: Re: [Cod-bugs] trouble in sign-up Dear Yosuke, On 2025-11-21 04:09, TANI Yosuke wrote: > I tried to sign up my account from following page, but I haven't receive > an email to confirm. > https://www.crystallography.net/cod/check_user.php?signup > In case your confirmation email did not arrive, here is the link for your account activation: https://www.crystallography.net/cod/check_user.php?activationstr=b4581f488e82a3138d395eadc1d17626 Clicking on this link should activate your account. Please let us know should you run into any additional obstacles. Best wishes, Andrius -- Andrius Merkys Vilnius University Institute of Biotechnology, Saul?tekio al. 7 LT-10257 Vilnius, Lithuania -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrius.merkys at gmail.com Fri Nov 21 14:29:24 2025 From: andrius.merkys at gmail.com (Andrius Merkys) Date: Fri, 21 Nov 2025 14:29:24 +0200 Subject: [Cod-bugs] trouble in sign-up In-Reply-To: References: <02e777fb-7519-45e1-ae2f-e8a01ea74282@gmail.com> Message-ID: Dear Yosuke, On 2025-11-21 11:23, TANI Yosuke wrote: > Thank you very much for your prompt reply. > I have?successfully logged in my account. Glad to hear this worked. > I've just deposited a structure #3000627, whose data was collected by > MicroED. > We refined the atomic model only kinematically, ignoring effects of > multiple scattering (dynamical diffraction). > This led to high R1 and wR2 values. > To deposite this data, I deleted following lines from the cif file: > > _refine_ls_R_factor_gt ? ? ? ? ? 0.1704 > _refine_ls_wR_factor_ref ? ? ? ? 0.4086 > > Could you please add these lines to the structure #3000627? I have just inserted these lines into the CIF file for COD structure 3000627. Thanks a lot for your contribution to the COD! Andrius -- Andrius Merkys Vilnius University Institute of Biotechnology, Saul?tekio al. 7 LT-10257 Vilnius, Lithuania -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From tani.yosuke.y1 at f.mail.nagoya-u.ac.jp Fri Nov 21 15:05:49 2025 From: tani.yosuke.y1 at f.mail.nagoya-u.ac.jp (TANI Yosuke) Date: Fri, 21 Nov 2025 13:05:49 +0000 Subject: [Cod-bugs] trouble in sign-up In-Reply-To: References: <02e777fb-7519-45e1-ae2f-e8a01ea74282@gmail.com> Message-ID: Dear Andrius, Thank you again for your kind support! Best regards, Yosuke ________________________________ ???: Andrius Merkys ????: 2025?11?21? 21:29 ??: TANI Yosuke CC: cod-bugs at ibt.lt ??: Re: [Cod-bugs] trouble in sign-up Dear Yosuke, On 2025-11-21 11:23, TANI Yosuke wrote: > Thank you very much for your prompt reply. > I have successfully logged in my account. Glad to hear this worked. > I've just deposited a structure #3000627, whose data was collected by > MicroED. > We refined the atomic model only kinematically, ignoring effects of > multiple scattering (dynamical diffraction). > This led to high R1 and wR2 values. > To deposite this data, I deleted following lines from the cif file: > > _refine_ls_R_factor_gt 0.1704 > _refine_ls_wR_factor_ref 0.4086 > > Could you please add these lines to the structure #3000627? I have just inserted these lines into the CIF file for COD structure 3000627. Thanks a lot for your contribution to the COD! Andrius -- Andrius Merkys Vilnius University Institute of Biotechnology, Saul?tekio al. 7 LT-10257 Vilnius, Lithuania -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -------------- next part -------------- An HTML attachment was scrubbed... URL: