From support at profex-xrd.org  Fri Jan 24 15:21:09 2025
From: support at profex-xrd.org (=?UTF-8?Q?Nicola_D=C3=B6belin?=)
Date: Fri, 24 Jan 2025 14:21:09 +0100
Subject: [Cod-bugs] COD to sqlite3 error
Message-ID: <f9f73af3-3ba8-4821-9b1a-843c38b01127@profex-xrd.org>

Hello,

I am the programmer of the Rietveld refinement software Profex. In the 
past few years, I used to convert the COD to sqlite3 to bundle it with 
Profex. For the conversion I used the attached bash script, which I 
created based on the information given on the COD wiki:

https://wiki.crystallography.net/creatingSQLdatabase/

However, this year the script no longer works and generates the 
following error message:

#-- begin output --

updating svn
Updating '.':
At revision 297460.
converting mysql to sqlite

 ????? ERROR (line 13): Invalid statement: Was expecting comment, or use, or
 ?????????????????????? set, or drop, or create, or alter, or insert, or
 ?????????????????????? delimiter, or empty statement
Error: translate: Error with parser 'SQL::Translator::Parser::MySQL':? 
no results at /usr/bin/sqlt line 353.
replacing escape sequences
importing data
CREATE TABLE data(...) failed: duplicate column name: 90
creating cod-250124.db
compressing
 ? adding: cod-250124.db3 (stored 0%)

#-- end output --

The resulting *.db3 file is empty. I always update the sqlite3 COD 
version in January, so the break must have occurred some time between 
Jan 2024 and Jan 2025. I also tried to run the script on an older Ubuntu 
20.04 system on which it used to work in the past, but to no avail. 
Since I am not familiar with MySQL and have very limited experience with 
sqlite, I would like to ask for help to create an updated sqlite version 
of the COD that I can bundle with Profex.

Any help will be appreciated.
Kind regards,
Nicola

-- 
Nicola D?belin
Web: https://www.profex-xrd.org
E-Mail: support at profex-xrd.org

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: update-and-convert-cod.sh
Type: application/x-shellscript
Size: 507 bytes
Desc: not available
URL: <http://lists.crystallography.net/pipermail/cod-bugs/attachments/20250124/2b7001d4/attachment.bin>

From antanas.vaitkus90 at gmail.com  Fri Jan 24 22:23:13 2025
From: antanas.vaitkus90 at gmail.com (Antanas Vaitkus)
Date: Fri, 24 Jan 2025 22:23:13 +0200
Subject: [Cod-bugs] COD to sqlite3 error
In-Reply-To: <f9f73af3-3ba8-4821-9b1a-843c38b01127@profex-xrd.org>
References: <f9f73af3-3ba8-4821-9b1a-843c38b01127@profex-xrd.org>
Message-ID: <CALHYoX4B1CdTjTiu18ZHL0MDg-xowWAKAUHq7BEghyKzWZUMPA@mail.gmail.com>

Dear Nicola,

The issue seems to arise due to inline comments output by MariaDB,
(e.g. "`time` time /* mariadb-5.3 */ DEFAULT NULL,") since sqlt MySQL
parser does not know how to handle them.

I have updated the script you sent to circumvent the issue. Please give
it a try and let us know if this fully solves your issue. The script does
work correctly on my machine.

If this solution works for you, I will update the documentation accordingly.

Sincerely,
Antanas

On Fri, 24 Jan 2025 at 15:22, Nicola D?belin <support at profex-xrd.org> wrote:

> Hello,
>
> I am the programmer of the Rietveld refinement software Profex. In the
> past few years, I used to convert the COD to sqlite3 to bundle it with
> Profex. For the conversion I used the attached bash script, which I
> created based on the information given on the COD wiki:
>
> https://wiki.crystallography.net/creatingSQLdatabase/
>
> However, this year the script no longer works and generates the
> following error message:
>
> #-- begin output --
>
> updating svn
> Updating '.':
> At revision 297460.
> converting mysql to sqlite
>
>        ERROR (line 13): Invalid statement: Was expecting comment, or use,
> or
>                         set, or drop, or create, or alter, or insert, or
>                         delimiter, or empty statement
> Error: translate: Error with parser 'SQL::Translator::Parser::MySQL':
> no results at /usr/bin/sqlt line 353.
> replacing escape sequences
> importing data
> CREATE TABLE data(...) failed: duplicate column name: 90
> creating cod-250124.db
> compressing
>    adding: cod-250124.db3 (stored 0%)
>
> #-- end output --
>
> The resulting *.db3 file is empty. I always update the sqlite3 COD
> version in January, so the break must have occurred some time between
> Jan 2024 and Jan 2025. I also tried to run the script on an older Ubuntu
> 20.04 system on which it used to work in the past, but to no avail.
> Since I am not familiar with MySQL and have very limited experience with
> sqlite, I would like to ask for help to create an updated sqlite version
> of the COD that I can bundle with Profex.
>
> Any help will be appreciated.
> Kind regards,
> Nicola
>
> --
> Nicola D?belin
> Web: https://www.profex-xrd.org
> E-Mail: support at profex-xrd.org
>
> --
> This message has been scanned for viruses and
> dangerous content by MailScanner, and is
> believed to be clean.
>
> _______________________________________________
> Cod-bugs mailing list
> Cod-bugs at lists.crystallography.net
> http://lists.crystallography.net/cgi-bin/mailman/listinfo/cod-bugs
>


-- 
Antanas Vaitkus,
Vilnius University,
Life Sciences Center,
Institute of Biotechnology,
room C521, Saul?tekio al. 7,
LT-10257 Vilnius, Lithuania

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.crystallography.net/pipermail/cod-bugs/attachments/20250124/923fb02d/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: update-and-convert-cod_2025-01-24.sh
Type: application/x-shellscript
Size: 625 bytes
Desc: not available
URL: <http://lists.crystallography.net/pipermail/cod-bugs/attachments/20250124/923fb02d/attachment.bin>