126: infinite import on merged svn-repository

(150)
John Reese (administrator)
2010-04-02 14:11

Will investigate.

(155)
obones (reporter)
2010-04-08 04:51

I have the exact same behavior, and it's even worse. There's no need to import again, the loop occurs with a very small number of revisions.
In my case, it imports revisions 1 through 201, then starts again at 1 until it errors out on an "out of memory" error. I don't even have to restart it, it loops on its own.
After it crashed, I had more than 100000 rows in the changeset table, all referring to revisions 1 through 201
Applying the proposed fixed above sorted the situation out, all revisions are now imported just fine.

Just for background, the repository has more than 43800 revisions and is the result of the merge of about 10 separate dumps made from another SCM system

(159)
John Reese (administrator)
2010-04-08 12:16

My biggest concern about the proposed fix is whether that SQL code is portable across implementations. Ie, will it work correctly on MySQL, Postgres, and MSSQL at the very least?

(162)
obones (reporter)
2010-04-08 12:38

Well, that syntax works under those three as it is part of the SQL standard.
The "unsigned" keyword is MySQL specific though, so it should be better to use "integer" instead.

(163)
John Reese (administrator)
2010-04-08 12:47

That's exactly the problem: "unsigned" only works in Mysql, but just "integer" doesn't. It seems that even the ANSI standard CAST() isn't portable enough due to differences in data types... =\ Either way, I've committed the fix for Mysql to a separate branch "bug126" that I've pushed to the public repository. At least for the time being, I'll try to keep that in sync with master until I can find a better solution.

(165)
obones (reporter)
2010-04-08 12:52

According to the following page, Integer should work all by itself:

http://dev.mysql.com/doc/refman/5.0/en/numeric-types.html [^]

(166)
John Reese (administrator)
2010-04-08 12:56

Database query failed. Error received from database was #1064: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'INTEGER ) DESC LIMIT 1' at line 3 for the query: SELECT revision FROM mantis_plugin_Source_changeset_table
WHERE repo_id='5'
ORDER BY CAST( revision AS INTEGER ) DESC.

(167)
obones (reporter)
2010-04-08 13:08

Yeah, you're right, the list of types supported by cast is not the list of types supported for a column type... That sucks.

(172)
as (reporter)
2010-04-09 06:52

That is what I saw, too. So I just made it work for me for my current installation and that is MySQL as you can see.
But I'm "happy" to see that I'm not the only one with this problem. Sorry obones ;)

Can we fix this with DECIMAL instead of int?

The cast to decimal should work for MySQL (http://dev.mysql.com/doc/refman/5.1/en/cast-functions.html [^]) and for MSSQL it seems to work also. Postgres I don't know.

I have not tested this with the plugin if decimal would work...

(173)
John Reese (administrator)
2010-04-09 07:15

Fix committed to master branch at 2010-04-08 19:35:52.

Related Changesets
Source Integration: master b9ef2021 Timestamp: 2010-04-08 19:35:52 Author: John Reese Committer: John Reese [ Details ] [ Diff ]	Fix 126: Sort SVN changesets by numeric revision To better handle merged SVN repositories, it is necessary to retrieve the latest changeset from the database based on integer revision number rather than by timestamp. This will prevent an "infinite loop" of repeatedly importing the same changesets in cases where the latest revisions were actually committed before earlier changesets. Using ANSI SQL's `CAST(? AS DECIMAL)` as it seems to be compatible with the most database platforms.
	mod - SourceSVN/SourceSVN.php	[ Diff ] [ File ]

Notes
(150) John Reese (administrator) 2010-04-02 14:11	Will investigate.

(155) obones (reporter) 2010-04-08 04:51	I have the exact same behavior, and it's even worse. There's no need to import again, the loop occurs with a very small number of revisions. In my case, it imports revisions 1 through 201, then starts again at 1 until it errors out on an "out of memory" error. I don't even have to restart it, it loops on its own. After it crashed, I had more than 100000 rows in the changeset table, all referring to revisions 1 through 201 Applying the proposed fixed above sorted the situation out, all revisions are now imported just fine. Just for background, the repository has more than 43800 revisions and is the result of the merge of about 10 separate dumps made from another SCM system

(159) John Reese (administrator) 2010-04-08 12:16	My biggest concern about the proposed fix is whether that SQL code is portable across implementations. Ie, will it work correctly on MySQL, Postgres, and MSSQL at the very least?

(162) obones (reporter) 2010-04-08 12:38	Well, that syntax works under those three as it is part of the SQL standard. The "unsigned" keyword is MySQL specific though, so it should be better to use "integer" instead.

(163) John Reese (administrator) 2010-04-08 12:47	That's exactly the problem: "unsigned" only works in Mysql, but just "integer" doesn't. It seems that even the ANSI standard CAST() isn't portable enough due to differences in data types... =\ Either way, I've committed the fix for Mysql to a separate branch "bug126" that I've pushed to the public repository. At least for the time being, I'll try to keep that in sync with master until I can find a better solution.

(165) obones (reporter) 2010-04-08 12:52	According to the following page, Integer should work all by itself: http://dev.mysql.com/doc/refman/5.0/en/numeric-types.html [^]

(166) John Reese (administrator) 2010-04-08 12:56	Database query failed. Error received from database was #1064: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'INTEGER ) DESC LIMIT 1' at line 3 for the query: SELECT revision FROM mantis_plugin_Source_changeset_table WHERE repo_id='5' ORDER BY CAST( revision AS INTEGER ) DESC.

(167) obones (reporter) 2010-04-08 13:08	Yeah, you're right, the list of types supported by cast is not the list of types supported for a column type... That sucks.

(172) as (reporter) 2010-04-09 06:52	That is what I saw, too. So I just made it work for me for my current installation and that is MySQL as you can see. But I'm "happy" to see that I'm not the only one with this problem. Sorry obones ;) Can we fix this with DECIMAL instead of int? The cast to decimal should work for MySQL (http://dev.mysql.com/doc/refman/5.1/en/cast-functions.html [^]) and for MSSQL it seems to work also. Postgres I don't know. I have not tested this with the plugin if decimal would work...

(173) John Reese (administrator) 2010-04-09 07:15	Fix committed to master branch at 2010-04-08 19:35:52.

Issue History
Date Modified	Username	Field	Change
2010-04-01 08:20	as	New Issue
2010-04-02 14:11	John Reese	Note Added: 150
2010-04-02 14:11	John Reese	Assigned To	=> John Reese
2010-04-02 14:11	John Reese	Status	new => acknowledged
2010-04-08 04:51	obones	Note Added: 155
2010-04-08 12:16	John Reese	Note Added: 159
2010-04-08 12:38	obones	Note Added: 162
2010-04-08 12:47	John Reese	Note Added: 163
2010-04-08 12:52	obones	Note Added: 165
2010-04-08 12:56	John Reese	Note Added: 166
2010-04-08 13:08	obones	Note Added: 167
2010-04-09 06:52	as	Note Added: 172
2010-04-09 07:15	John Reese	Note Added: 173
2010-04-09 07:15	John Reese	Status	acknowledged => resolved
2010-04-09 07:15	John Reese	Fixed in Version	=> 0.16
2010-04-09 07:15	John Reese	Resolution	open => fixed
2010-04-09 07:15	John Reese	Changeset attached	master b9ef2021 =>

Relationships