[Cuis-dev] fileout. proposed 2 new methods for strict file chunks reading
Nicola Mingotti
nmingotti at gmail.com
Thu Oct 21 08:47:32 PDT 2021
Hi Hernan,
. forget the code and test. I can rewrite it from scratch with test. I
actually changed
existing code for "politeness" ;)
. for me it is very important to have this matter fixed, well and for
the future.
It is not good to have standard lib functionality disseminated in my
application packages.
. since I found Linux stdlib has a function to do well what i want i
will use that name(s)
to avoid confusion and recycle already existing function names.
"getline" and "getdelim".
. if you really dislike this functions I can put them in OSProcess and
maybe
just link the C version only for Linux/BSD. So much I think they are
valuable in the server environment.
. to fix this i need maybe 1-2 days. If i need to link the C functions I
don't know, since I never tried.
So, let me know, if you are not against these functions I am open to
implement them well.
===== Extra considerations whose reading is secondary ==================
. your fix was one step in the right direction but not enough, you also
need to
bring back the stream pointer to the last existant $A. This is to say:
too complex.
A good method must do all its chore, not leave us back the dirty
business and special conditions.
. I understand the concision, small core etc. On the other side, i
run Cuis on the servers. the most important thing there is on servers
are files and
sockets. You must read from there all of the time. It must be easy and
idiot proof,
rock solid and resistant to concurrent processing as far as possible.
. I see that Python and Ruby standard library do it wrong, at bit better
than Cuis 'upTo' does.
but still bad. They leave you the '\n' at the end, but, if any process
goes on writing
'f1.txt' Ruby and Python lost the half backed record !
-------- Linux
$> printf 'line-1\nline-2\nline-TRAP' > f1.txt
# python
$> python3.9 -c "f=open('f1.txt','r'); print(f.readlines())"
=> ['line-1\n', 'line-2\n', 'line-TRAP']
# ruby
$> ruby -e "f=open('f1.txt','r'); puts f.readlines().to_s; "
=> ["line-1\n", "line-2\n", "line-TRAP"]
# both Python and Ruby ate the half backed record ! bad !
---------------------------------------------------------
. C and CommonLisp standard libraries have a way to do it right:
-) CL read-line.
http://www.lispworks.com/documentation/HyperSpec/Body/f_rd_lin.htm#read-line
-) C getline. https://man7.org/linux/man-pages/man3/getline.3.html
. I understand I am probably the only one running Cuis in the server so
I am the first
to step into a few particular problems.
. In my opinion Cuis in the Server can be a good match, up to now i have
2 small
company services working and a big one project in continuous development.
Time will tell. Sturdiness, undertandability and ease of modification
were my top priority.
Up to now things are at least working.
======================================================
bye
Nicola
On 10/21/21 14:53, Hernan Wilkinson wrote:
> Hi Nicola,
> I see your point regarding the functionality of upTo:, but you can
> easily overcome that using #peekBack. Using you example:
> -----
> s _ 'hello-1Ahello-2Ahel'.
> '/tmp/test.txt' asFileEntry fileContents: s.
>
> st1 _ '/tmp/test.txt' asFileEntry readStream .
>
> st1 upTo: $A. " 'hello-1' "
> st1 upTo: $A. " 'hello-2' "
> st1 upTo: $A. " 'hel' "
> (st1 atEnd and: [ st1 peekBack ~= $A ]) ifTrue: [ self error: 'End of
> file without delimiter ].
> ------
> Regarding my concern of adding this functionality to Cuis, we are
> trying to have a compact set of classes and methods to reduce
> complexity (or at least not increase it) and help newcomers to
> understand it and oldies to remember it :-) . We are also trying to
> add more and more tests because it is the only way to keep a system
> from becoming a legacy one and to reduce the fear it produces to
> change something.
> The strictUpTo:startPos: you are sending is almost a copy of the
> upTo: method, with a few lines changed. Even though the functionality
> makes sense (although right now you are the only one needing it and as
> I said, you can use peekBack to overcome it), adding that method adds
> repeated code which in the long term makes it more difficult to
> understand and maintain, even more because it does not have tests.
> So I hope you understand that as maintainers of Cuis, we want to be
> loyal to the goals I mentioned before and keep Cuis as clean and
> simple as possible. If you can refactor what you sent to avoid having
> repeated code with #upTo: and add tests that verify the functionality
> of both methods (strictUpTo: and upTo:), that will make our task
> easier and meet the goals we have. If you think this does not make
> sense to you, or you do not have the time to do it, it is completely
> understandable and in that case I suggest for you to have it as an
> extension of the StandardFileStream class or just use the peekBack
> message as I showed.
> I hope you understand my concern and agree with me. If not, please
> let me know.
>
> Cheers!
> Hernan.
>
>
> On Tue, Oct 19, 2021 at 10:32 AM Nicola Mingotti <nmingotti at gmail.com>
> wrote:
>
>
> Hi Hernan,
>
> In all frankness, in I would wipe out the old 'upTo' because its
> behavior is a bit "wild".
>
> On the other side, I understand it may create problems in
> retro-compatibility, that is why for
> the moment i propose to add a new method which behaves a bit better.
>
> I hope this example explains the problem:
> -------------------------------------------------------
> s _ 'hello-1Ahello-2Ahel'.
> '/tmp/test.txt' asFileEntry fileContents: s.
>
> st1 _ '/tmp/test.txt' asFileEntry readStream .
>
> st1 upTo: $A. " 'hello-1' "
> st1 upTo: $A. " 'hello-2' "
> st1 upTo: $A. " 'hel' " "(*)"
> ------------------------------------------------------
> (*) You can't establish in any way if you actually found an "A"
> terminated block or just hit the end of file
> (*) If you hit the end of file you eat an incomplete record, this
> is another problem, maybe another process
> was going to end writing that record but you will never know.
>
> Maybe there is another method around that performs similarly to
> 'strictUpTp', if there is I did not find it, sorry.
>
> IMHO, In a scale of importance from 0 to 10, this method, for a
> programmer, >= 8.
> I would definitely not put it into an external package, too much
> fundamental.
>
> bye
> Nicola
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On 10/19/21 14:44, Hernan Wilkinson wrote:
>> Hi Nicola!
>> I was wondering, why are you suggesting adding them to the base?
>> Is it not enough to implement them as an extension in your package?
>> Also, I think that any new functionality should come with its
>> corresponding tests to help the maintenance and understanding of
>> the functionality.
>>
>> Cheers!
>> Hernan.
>>
>>
>> On Tue, Oct 19, 2021 at 7:04 AM Nicola Mingotti via Cuis-dev
>> <cuis-dev at lists.cuis.st> wrote:
>>
>> Hi Juan, guys,
>>
>> I would like to add to Cuis the 2 methods i attach here. One
>> is a helper method.
>>
>> -----------
>> StandardFileStream strictUpTo: delim.
>> -----------
>>
>> Differently from 'upTo: delim' this method:
>> 1. Does not return stuff if it does not find 'delim'.
>> 2. Does not upgrade the position on the stream if does not
>> find 'delim'.
>> 3. If it finds 'delim' returns a chunk that includes it.
>>
>> I am parsing log files at the moment, this is very much useful.
>>
>> NOTE. Up to now I tested only on small files.
>>
>> bye
>> Nicola
>>
>>
>>
>>
>> --
>> Cuis-dev mailing list
>> Cuis-dev at lists.cuis.st
>> https://lists.cuis.st/mailman/listinfo/cuis-dev
>>
>>
>>
>> --
>> <https://10pines.com/>
>>
>>
>> Hernán Wilkinson
>>
>>
>> Software Developer & Coach
>>
>> Alem 896, Floor 6, Buenos Aires, Argentina
>>
>> +54 11 6091 3125
>>
>> @HernanWilkinson
>>
>
>
>
> --
> <https://10pines.com/>
>
>
> Hernán Wilkinson
>
>
> Software Developer & Coach
>
> Alem 896, Floor 6, Buenos Aires, Argentina
>
> +54 11 6091 3125
>
> @HernanWilkinson
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cuis.st/mailman/archives/cuis-dev/attachments/20211021/dc2c76ad/attachment-0001.htm>
More information about the Cuis-dev
mailing list