This removing-nulls-code is invoked only if the read line of a file contains nulls. I believe that all previously loaded files
did not contain nulls. So while you do not load P-Cad file (or some other defected file), this part of code will not run and
thus there is no performance degradation.
Moreover, fgets (previous) implementation of FILE_LINE_READER does not work if it still meets a line with null in it.

"Summary, I don't like any of the changes to richio.  You are not fixing bugs, you have
made it much slower."


It does not look like bug fixing. It is a workaround at least for P-Cad files.
As I said it does not affect the previous world. In case of P-Cad files, that memory move operation occurs only 1 time
in only 1 of 5 lines on average. Taking into account that file operation (getline) takes more time compared to memmove,
I think that removing the nulls in P-Cad files may add maximum 10% of time to process the whole file.
If it is critical or even files exist with much more nulls in them (which I have not seen so far), I hope we can replace '\0'
occurrences with spaces instead of removing them.

"Once you go down the size_t road, you may have to stay on that road, co-mingling with int and unsigned then require a promotion for everything to a common primitive size."

I do not insist on using size_t. I do not believe that lines can be so long. We can limit the result of getline down to
unsigned int and do not use size_t.

"Let me know what the problem is, and I can offer a better solution."

Please, find the enclosed file 1line.pcb. I created it artificially as an example of the problem.
I see two ways so far:
1) to use getline
2) to write own implementation

As Jean-Pierre noticed about problem with getline in Mingw, first what comes to my mind is to use the source code of
getline and put it as a conditional compilation for Mingw.

May be there are other suggestions.

________________________________
From: Dick Hollenbeck <email address hidden>
To: <email address hidden> 
Sent: Sunday, October 7, 2012 6:13 PM
Subject: Re: [Merge] lp:~al-lunev/kicad/pcad2kicad-pcb-plugin-only into lp:kicad

+        // remove all nulls inside a string (e.g. ACCEL/P-Cad ASCII files often contain
nulls)
+        while( (ssize_t)strlen( line ) < actual_len )
+        {
+            str_ptr = strchr( line, '\0' );
+            null_pos = str_ptr - line;
+            if( str_ptr )
+            {
+                // left shift the substring starting right after the fake null position
+                // and ending at the real useful null position
+                memmove( str_ptr, str_ptr + 1, actual_len - null_pos );
+                actual_len--;
+            }
+        }



class LINE_READER

First, the intention dubious at best:  why spend CPU time in a *general purpose, widely
used function* to remove nulls?  This is like taking your screwdriver and duct taping a
hammer onto the end of it, just for one project.  No welding on the hammer, such that now
you pay a price for rotating the screwdriver on other projects.  The screwdriver is used
by others, not just this obscure conversion program.


Second, the implementation is bad, terribly inefficent.  But that is moot, since the
intention is a bug in and of itself.



size_t is 64 bit on gcc x86_64 linux.  int and unsigned are 32 bit.

Once you go down the size_t road, you may have to stay on that road, co-mingling with int
and unsigned then require a promotion for everything to a common primitive size.


We regularly use int and unsigned for loops, not size_t.


Summary, I don't like any of the changes to richio.  You are not fixing bugs, you have
made it much slower.


Where are these nuls a problem?  Is it in DSNLEXER?  If so, maybe we should be paying
attention to line length there rather than nul-ness.  Surely we can skip over nuls there,
or treat them like whitespace if that is what is needed.

I do not believe \0 is a valid UTF8 character, so maybe a tweak to DSNLEXER::NextTok() is
better.   Let me know what the problem is, and I can offer a better solution.







-- 
https://code.launchpad.net/~al-lunev/kicad/pcad2kicad-pcb-plugin-only/+merge/121318
You are the owner of lp:~al-lunev/kicad/pcad2kicad-pcb-plugin-only.