[Cuis-dev] fileout. proposed 2 new methods for strict file chunks reading

Nicola Mingotti nmingotti at gmail.com
Thu Oct 21 10:32:17 PDT 2021


Hi Hernan,

ok, let me try, it is too many days i am talking about it.

I will let you know soon

bye
Nicola




On 10/21/21 19:02, Hernan Wilkinson wrote:
> Hi Nicolas,
>  if you could refactor upTo: to use the same code as strictUpTo: and 
> write the tests to check that everything works as expected, that would 
> be great!
>  I would not use the names of the Linux stdlib for those messages nor 
> the C functions, it is not necessary...
>  If you do not have the time to do it, I can give it a try if you wish.
>
> Cheers!
> Hernan.
>
> On Thu, Oct 21, 2021 at 12:47 PM Nicola Mingotti <nmingotti at gmail.com> 
> wrote:
>
>
>     Hi Hernan,
>
>     . forget the code and test. I can rewrite it from scratch with
>     test. I actually changed
>     existing code for "politeness" ;)
>
>     . for me it is very important to have this matter fixed, well and
>     for the future.
>     It is not good to have standard lib functionality disseminated in
>     my application packages.
>
>     . since I found Linux stdlib has a function to do well what i want
>     i will use that name(s)
>     to avoid confusion and recycle already existing function names.
>     "getline" and "getdelim".
>
>     . if you really dislike this functions I can put them in OSProcess
>     and maybe
>     just link the C version only for Linux/BSD. So much I think they
>     are valuable in the server environment.
>
>     . to fix this i need maybe 1-2 days. If i need to link the C
>     functions I don't know, since I never tried.
>
>     So, let me know, if you are not against these functions I am open
>     to implement them well.
>
>
>     ===== Extra considerations whose reading is secondary
>     ==================
>
>     . your fix was one step in the right direction but not enough, you
>     also need to
>     bring back the stream pointer to the last existant $A. This is to
>     say: too complex.
>     A good method must do all its chore, not leave us back the dirty
>     business and special conditions.
>
>     . I understand the concision, small core etc. On the other side, i
>     run Cuis on the servers.  the most important thing there is on
>     servers are files and
>     sockets. You must read from there all of the time. It must be easy
>     and idiot proof,
>     rock solid and resistant to concurrent processing as far as possible.
>
>     . I see that Python and Ruby standard library do it wrong, at bit
>     better than Cuis 'upTo' does.
>     but still bad. They leave you the '\n' at the end, but, if any
>     process goes on writing
>     'f1.txt' Ruby and Python lost the half backed record !
>     -------- Linux
>     $> printf 'line-1\nline-2\nline-TRAP' > f1.txt
>     # python
>     $> python3.9 -c "f=open('f1.txt','r'); print(f.readlines())"
>     => ['line-1\n', 'line-2\n', 'line-TRAP']
>     # ruby
>     $> ruby -e "f=open('f1.txt','r'); puts f.readlines().to_s;  "
>     => ["line-1\n", "line-2\n", "line-TRAP"]
>     # both Python and Ruby ate the half backed record ! bad !
>     ---------------------------------------------------------
>
>     . C and CommonLisp standard libraries have a way to do it right:
>     -) CL read-line.
>     http://www.lispworks.com/documentation/HyperSpec/Body/f_rd_lin.htm#read-line
>
>     -) C getline. https://man7.org/linux/man-pages/man3/getline.3.html
>
>     . I understand I am probably the only one running Cuis in the
>     server so I am the first
>     to step into a few particular problems.
>
>     . In my opinion Cuis in the Server can be a good match, up to now
>     i have 2 small
>     company services working and a big one project in continuous
>     development.
>     Time will tell. Sturdiness, undertandability and ease of
>     modification were my top priority.
>     Up to now things are at least working.
>
>     ======================================================
>
>     bye
>     Nicola
>
>
>
>
>
>
>
>     On 10/21/21 14:53, Hernan Wilkinson wrote:
>>     Hi Nicola,
>>      I see your point regarding the functionality of upTo:, but you
>>     can easily overcome that using #peekBack. Using you example:
>>     -----
>>     s _ 'hello-1Ahello-2Ahel'.
>>     '/tmp/test.txt' asFileEntry fileContents: s.
>>
>>     st1 _ '/tmp/test.txt' asFileEntry readStream .
>>
>>     st1 upTo: $A. " 'hello-1' "
>>     st1 upTo: $A. " 'hello-2' "
>>     st1 upTo: $A. " 'hel' "
>>     (st1 atEnd and: [ st1 peekBack ~= $A ]) ifTrue: [ self error:
>>     'End of file without delimiter ].
>>     ------
>>      Regarding my concern of adding this functionality to Cuis, we
>>     are trying to have a compact set of classes and methods to reduce
>>     complexity (or at least not increase it) and help newcomers to
>>     understand it and oldies to remember it :-) . We are also trying
>>     to add more and more tests because it is the only way to keep a
>>     system from becoming a legacy one and to reduce the fear it
>>     produces to change something.
>>      The strictUpTo:startPos: you are sending is almost a copy of the
>>     upTo: method, with a few lines changed. Even though the
>>     functionality makes sense (although right now you are the only
>>     one needing it and as I said, you can use peekBack to overcome
>>     it), adding that method adds repeated code which in the long term
>>     makes it more difficult to understand and maintain, even more
>>     because it does not have tests.
>>      So I hope you understand that as maintainers of Cuis, we want to
>>     be loyal to the goals I mentioned before and keep Cuis as clean
>>     and simple as possible. If you can refactor what you sent to
>>     avoid having repeated code with #upTo: and add tests that verify
>>     the functionality of both methods (strictUpTo: and upTo:), that
>>     will make our task easier and meet the goals we have. If you
>>     think this does not make sense to you, or you do not have the
>>     time to do it, it is completely understandable and in that case I
>>     suggest for you to have it as an extension of the
>>     StandardFileStream class or just use the peekBack message as I
>>     showed.
>>      I hope you understand my concern and agree with me. If not,
>>     please let me know.
>>
>>     Cheers!
>>     Hernan.
>>
>>
>>     On Tue, Oct 19, 2021 at 10:32 AM Nicola Mingotti
>>     <nmingotti at gmail.com> wrote:
>>
>>
>>         Hi Hernan,
>>
>>         In all frankness, in I would wipe out the old 'upTo' because
>>         its behavior is a bit "wild".
>>
>>         On the other side, I understand it may create problems in
>>         retro-compatibility, that is why for
>>         the moment i propose to add a new method which behaves a bit
>>         better.
>>
>>         I hope this example explains the problem:
>>         -------------------------------------------------------
>>         s _ 'hello-1Ahello-2Ahel'.
>>         '/tmp/test.txt' asFileEntry fileContents: s.
>>
>>         st1 _ '/tmp/test.txt' asFileEntry readStream .
>>
>>         st1 upTo: $A. " 'hello-1' "
>>         st1 upTo: $A. " 'hello-2' "
>>         st1 upTo: $A. " 'hel' "         "(*)"
>>         ------------------------------------------------------
>>         (*) You can't establish in any way if you actually found an
>>         "A" terminated block or just hit the end of file
>>         (*) If you hit the end of file you eat an incomplete record,
>>         this is another problem, maybe another process
>>         was going to end writing that record but you will never know.
>>
>>         Maybe there is another method around that performs similarly
>>         to 'strictUpTp', if there is I did not find it, sorry.
>>
>>         IMHO, In a scale of importance from 0 to 10, this method, for
>>         a programmer, >= 8.
>>         I would definitely not put it into an external package, too
>>         much fundamental.
>>
>>         bye
>>         Nicola
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>         On 10/19/21 14:44, Hernan Wilkinson wrote:
>>>         Hi Nicola!
>>>          I was wondering, why are you suggesting adding them to the
>>>         base? Is it not enough to implement them as an extension in
>>>         your package?
>>>          Also, I think that any new functionality should come with
>>>         its corresponding tests to help the maintenance and
>>>         understanding of the functionality.
>>>
>>>         Cheers!
>>>         Hernan.
>>>
>>>
>>>         On Tue, Oct 19, 2021 at 7:04 AM Nicola Mingotti via Cuis-dev
>>>         <cuis-dev at lists.cuis.st> wrote:
>>>
>>>             Hi Juan, guys,
>>>
>>>             I would like to add to Cuis the 2 methods i attach here.
>>>             One is a helper method.
>>>
>>>             -----------
>>>             StandardFileStream strictUpTo: delim.
>>>             -----------
>>>
>>>             Differently from 'upTo: delim' this method:
>>>             1. Does not return stuff if it does not find 'delim'.
>>>             2. Does not upgrade the position on the stream if does
>>>             not find 'delim'.
>>>             3. If it finds 'delim' returns a chunk that includes it.
>>>
>>>             I am parsing log files at the moment, this is very much
>>>             useful.
>>>
>>>             NOTE. Up to now I tested only on small files.
>>>
>>>             bye
>>>             Nicola
>>>
>>>
>>>
>>>
>>>             -- 
>>>             Cuis-dev mailing list
>>>             Cuis-dev at lists.cuis.st
>>>             https://lists.cuis.st/mailman/listinfo/cuis-dev
>>>
>>>
>>>
>>>         -- 
>>>         <https://10pines.com/>
>>>
>>>
>>>           Hernán Wilkinson
>>>
>>>
>>>             Software Developer & Coach
>>>
>>>         Alem 896, Floor 6, Buenos Aires, Argentina
>>>
>>>         +54 11 6091 3125
>>>
>>>         @HernanWilkinson
>>>
>>
>>
>>
>>     -- 
>>     <https://10pines.com/>
>>
>>
>>       Hernán Wilkinson
>>
>>
>>         Software Developer & Coach
>>
>>     Alem 896, Floor 6, Buenos Aires, Argentina
>>
>>     +54 11 6091 3125
>>
>>     @HernanWilkinson
>>
>
>
>
> -- 
> <https://10pines.com/>
>
>
>   Hernán Wilkinson
>
>
>     Software Developer & Coach
>
> Alem 896, Floor 6, Buenos Aires, Argentina
>
> +54 11 6091 3125
>
> @HernanWilkinson
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cuis.st/mailman/archives/cuis-dev/attachments/20211021/51f2349c/attachment-0001.htm>


More information about the Cuis-dev mailing list