[Cuis-dev] fileout. proposed 2 new methods for strict file chunks reading

Thu Oct 21 10:02:57 PDT 2021

Hi Nicolas,
 if you could refactor upTo: to use the same code as strictUpTo: and write
the tests to check that everything works as expected, that would be great!
 I would not use the names of the Linux stdlib for those messages nor the C
functions, it is not necessary...
 If you do not have the time to do it, I can give it a try if you wish.

Cheers!
Hernan.

On Thu, Oct 21, 2021 at 12:47 PM Nicola Mingotti <nmingotti at gmail.com>
wrote:

>
> Hi Hernan,
>
> . forget the code and test. I can rewrite it from scratch with test. I
> actually changed
> existing code for "politeness" ;)
>
> . for me it is very important to have this matter fixed, well and for the
> future.
> It is not good to have standard lib functionality disseminated in my
> application packages.
>
> . since I found Linux stdlib has a function to do well what i want i will
> use that name(s)
> to avoid confusion and recycle already existing function names. "getline"
> and "getdelim".
>
> . if you really dislike this functions I can put them in OSProcess and
> maybe
> just link the C version only for Linux/BSD. So much I think they are
> valuable in the server environment.
>
> . to fix this i need maybe 1-2 days. If i need to link the C functions I
> don't know, since I never tried.
>
> So, let me know, if you are not against these functions I am open to
> implement them well.
>
>
> ===== Extra considerations whose reading is secondary ==================
>
> . your fix was one step in the right direction but not enough, you also
> need to
> bring back the stream pointer to the last existant $A. This is to say: too
> complex.
> A good method must do all its chore, not leave us back the dirty business
> and special conditions.
>
> . I understand the concision, small core etc. On the other side, i
> run Cuis on the servers.  the most important thing there is on servers are
> files and
> sockets. You must read from there all of the time. It must be easy and
> idiot proof,
> rock solid and resistant to concurrent processing as far as possible.
>
> . I see that Python and Ruby standard library do it wrong, at bit better
> than Cuis 'upTo' does.
> but still bad. They leave you the '\n' at the end, but, if any process
> goes on writing
> 'f1.txt' Ruby and Python lost the half backed record !
> -------- Linux
> $> printf 'line-1\nline-2\nline-TRAP' > f1.txt
> # python
> $> python3.9 -c "f=open('f1.txt','r'); print(f.readlines())"
> => ['line-1\n', 'line-2\n', 'line-TRAP']
> # ruby
> $> ruby -e "f=open('f1.txt','r'); puts f.readlines().to_s;  "
> => ["line-1\n", "line-2\n", "line-TRAP"]
> # both Python and Ruby ate the half backed record ! bad !
> ---------------------------------------------------------
>
> . C and CommonLisp standard libraries have a way to do it right:
> -) CL read-line.
> http://www.lispworks.com/documentation/HyperSpec/Body/f_rd_lin.htm#read-line
> -) C getline. https://man7.org/linux/man-pages/man3/getline.3.html
>
> . I understand I am probably the only one running Cuis in the server so I
> am the first
> to step into a few particular problems.
>
> . In my opinion Cuis in the Server can be a good match, up to now i have 2
> small
> company services working and a big one project in continuous development.
> Time will tell. Sturdiness, undertandability and ease of modification were
> my top priority.
> Up to now things are at least working.
>
> ======================================================
>
> bye
> Nicola
>
>
>
>
>
>
>
> On 10/21/21 14:53, Hernan Wilkinson wrote:
>
> Hi Nicola,
>  I see your point regarding the functionality of upTo:, but you can easily
> overcome that using #peekBack. Using you example:
> -----
> s _ 'hello-1Ahello-2Ahel'.
> '/tmp/test.txt' asFileEntry fileContents: s.
>
> st1 _ '/tmp/test.txt' asFileEntry readStream .
>
> st1 upTo: $A. " 'hello-1' "
> st1 upTo: $A. " 'hello-2' "
> st1 upTo: $A. " 'hel' "
> (st1 atEnd and: [ st1 peekBack ~= $A ]) ifTrue: [ self error: 'End of file
> without delimiter ].
> ------
>
>  Regarding my concern of adding this functionality to Cuis, we are trying
> to have a compact set of classes and methods to reduce complexity (or at
> least not increase it) and help newcomers to understand it and oldies to
> remember it :-) . We are also trying to add more and more tests because it
> is the only way to keep a system from becoming a legacy one and to reduce
> the fear it produces to change something.
>  The strictUpTo:startPos: you are sending is almost a copy of the upTo:
> method, with a few lines changed. Even though the functionality makes sense
> (although right now you are the only one needing it and as I said, you can
> use peekBack to overcome it), adding that method adds repeated code which
> in the long term makes it more difficult to understand and maintain, even
> more because it does not have tests.
>  So I hope you understand that as maintainers of Cuis, we want to be loyal
> to the goals I mentioned before and keep Cuis as clean and simple as
> possible. If you can refactor what you sent to avoid having repeated code
> with #upTo: and add tests that verify the functionality of both methods
> (strictUpTo: and upTo:), that will make our task easier and meet the goals
> we have. If you think this does not make sense to you, or you do not have
> the time to do it, it is completely understandable and in that case I
> suggest for you to have it as an extension of the StandardFileStream class
> or just use the peekBack message as I showed.
>  I hope you understand my concern and agree with me. If not, please let me
> know.
>
> Cheers!
> Hernan.
>
>
> On Tue, Oct 19, 2021 at 10:32 AM Nicola Mingotti <nmingotti at gmail.com>
> wrote:
>
>>
>> Hi Hernan,
>>
>> In all frankness, in I would wipe out the old 'upTo' because its behavior
>> is a bit "wild".
>>
>> On the other side, I understand it may create problems in
>> retro-compatibility, that is why for
>> the moment i propose to add a new method which behaves a bit better.
>>
>> I hope this example explains the problem:
>> -------------------------------------------------------
>> s _ 'hello-1Ahello-2Ahel'.
>> '/tmp/test.txt' asFileEntry fileContents: s.
>>
>> st1 _ '/tmp/test.txt' asFileEntry readStream .
>>
>> st1 upTo: $A. " 'hello-1' "
>> st1 upTo: $A. " 'hello-2' "
>> st1 upTo: $A. " 'hel' "         "(*)"
>> ------------------------------------------------------
>> (*) You can't establish in any way if you actually found an "A"
>> terminated block or just hit the end of file
>> (*) If you hit the end of file you eat an incomplete record, this is
>> another problem, maybe another process
>> was going to end writing that record but you will never know.
>>
>> Maybe there is another method around that performs similarly to
>> 'strictUpTp', if there is I did not find it, sorry.
>>
>> IMHO, In a scale of importance from 0 to 10, this method, for a
>> programmer, >= 8.
>> I would definitely not put it into an external package, too much
>> fundamental.
>>
>> bye
>> Nicola
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On 10/19/21 14:44, Hernan Wilkinson wrote:
>>
>> Hi Nicola!
>>  I was wondering, why are you suggesting adding them to the base? Is it
>> not enough to implement them as an extension in your package?
>>  Also, I think that any new functionality should come with its
>> corresponding tests to help the maintenance and understanding of the
>> functionality.
>>
>> Cheers!
>> Hernan.
>>
>>
>> On Tue, Oct 19, 2021 at 7:04 AM Nicola Mingotti via Cuis-dev <
>> cuis-dev at lists.cuis.st> wrote:
>>
>>> Hi Juan, guys,
>>>
>>> I would like to add to Cuis the 2 methods i attach here. One is a helper
>>> method.
>>>
>>> -----------
>>> StandardFileStream strictUpTo: delim.
>>> -----------
>>>
>>> Differently from 'upTo: delim' this method:
>>> 1. Does not return stuff if it does not find 'delim'.
>>> 2. Does not upgrade the position on the stream if does not find 'delim'.
>>> 3. If it finds 'delim' returns a chunk that includes it.
>>>
>>> I am parsing log files at the moment, this is very much useful.
>>>
>>> NOTE. Up to now I tested only on small files.
>>>
>>> bye
>>> Nicola
>>>
>>>
>>>
>>>
>>> --
>>> Cuis-dev mailing list
>>> Cuis-dev at lists.cuis.st
>>> https://lists.cuis.st/mailman/listinfo/cuis-dev
>>>
>>
>>
>> --
>> <https://10pines.com/> Hernán Wilkinson Software Developer & Coach
>>
>> Alem 896, Floor 6, Buenos Aires, Argentina
>>
>> +54 11 6091 3125
>>
>> @HernanWilkinson
>>
>>
>>
>
> --
> <https://10pines.com/> Hernán Wilkinson Software Developer & Coach
>
> Alem 896, Floor 6, Buenos Aires, Argentina
>
> +54 11 6091 3125
>
> @HernanWilkinson
>
>
>

-- 
<https://10pines.com/>Hernán WilkinsonSoftware Developer & Coach

Alem 896, Floor 6, Buenos Aires, Argentina

+54 11 6091 3125

@HernanWilkinson
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cuis.st/mailman/archives/cuis-dev/attachments/20211021/9a237804/attachment-0001.htm>