[Cuis-dev] fileout. proposed 2 new methods for strict file chunks reading
Nicola Mingotti
nmingotti at gmail.com
Fri Oct 22 08:18:36 PDT 2021
Hi Hernan,
We will have opportunity to work together on larger problems, this is
too small.
It would take more time to talk than to do things ;)
I have a proposed version. I rewrote the methods. wrote the test. I kept
a good part
of the original code which may have evolved for efficiency over time.
upToLegacy method can of course be eliminated. it is there only for
reference.
upTo: XXX --- now calls ---> upTo: XXX strict: false
upTo: XXX strict: XXX ------ is recursive, it needs an extra helper
method to remember a parameter (Scheme recursion style) ----> upTo: XXX
strict: XXX posMemo: xxxx
See attached fileout
bye
Nicola
On 10/21/21 19:49, Hernan Wilkinson wrote:
> ok, let me know. I wish we could do it together but my agenda (and I
> guess yours) is almost always full...
>
> On Thu, Oct 21, 2021 at 2:32 PM Nicola Mingotti <nmingotti at gmail.com>
> wrote:
>
>
> Hi Hernan,
>
> ok, let me try, it is too many days i am talking about it.
>
> I will let you know soon
>
> bye
> Nicola
>
>
>
>
> On 10/21/21 19:02, Hernan Wilkinson wrote:
>> Hi Nicolas,
>> if you could refactor upTo: to use the same code as strictUpTo:
>> and write the tests to check that everything works as expected,
>> that would be great!
>> I would not use the names of the Linux stdlib for those messages
>> nor the C functions, it is not necessary...
>> If you do not have the time to do it, I can give it a try if you
>> wish.
>>
>> Cheers!
>> Hernan.
>>
>> On Thu, Oct 21, 2021 at 12:47 PM Nicola Mingotti
>> <nmingotti at gmail.com> wrote:
>>
>>
>> Hi Hernan,
>>
>> . forget the code and test. I can rewrite it from scratch
>> with test. I actually changed
>> existing code for "politeness" ;)
>>
>> . for me it is very important to have this matter fixed, well
>> and for the future.
>> It is not good to have standard lib functionality
>> disseminated in my application packages.
>>
>> . since I found Linux stdlib has a function to do well what i
>> want i will use that name(s)
>> to avoid confusion and recycle already existing function
>> names. "getline" and "getdelim".
>>
>> . if you really dislike this functions I can put them in
>> OSProcess and maybe
>> just link the C version only for Linux/BSD. So much I think
>> they are valuable in the server environment.
>>
>> . to fix this i need maybe 1-2 days. If i need to link the C
>> functions I don't know, since I never tried.
>>
>> So, let me know, if you are not against these functions I am
>> open to implement them well.
>>
>>
>> ===== Extra considerations whose reading is secondary
>> ==================
>>
>> . your fix was one step in the right direction but not
>> enough, you also need to
>> bring back the stream pointer to the last existant $A. This
>> is to say: too complex.
>> A good method must do all its chore, not leave us back the
>> dirty business and special conditions.
>>
>> . I understand the concision, small core etc. On the other
>> side, i
>> run Cuis on the servers. the most important thing there is
>> on servers are files and
>> sockets. You must read from there all of the time. It must be
>> easy and idiot proof,
>> rock solid and resistant to concurrent processing as far as
>> possible.
>>
>> . I see that Python and Ruby standard library do it wrong, at
>> bit better than Cuis 'upTo' does.
>> but still bad. They leave you the '\n' at the end, but, if
>> any process goes on writing
>> 'f1.txt' Ruby and Python lost the half backed record !
>> -------- Linux
>> $> printf 'line-1\nline-2\nline-TRAP' > f1.txt
>> # python
>> $> python3.9 -c "f=open('f1.txt','r'); print(f.readlines())"
>> => ['line-1\n', 'line-2\n', 'line-TRAP']
>> # ruby
>> $> ruby -e "f=open('f1.txt','r'); puts f.readlines().to_s; "
>> => ["line-1\n", "line-2\n", "line-TRAP"]
>> # both Python and Ruby ate the half backed record ! bad !
>> ---------------------------------------------------------
>>
>> . C and CommonLisp standard libraries have a way to do it right:
>> -) CL read-line.
>> http://www.lispworks.com/documentation/HyperSpec/Body/f_rd_lin.htm#read-line
>>
>> -) C getline.
>> https://man7.org/linux/man-pages/man3/getline.3.html
>>
>> . I understand I am probably the only one running Cuis in the
>> server so I am the first
>> to step into a few particular problems.
>>
>> . In my opinion Cuis in the Server can be a good match, up to
>> now i have 2 small
>> company services working and a big one project in continuous
>> development.
>> Time will tell. Sturdiness, undertandability and ease of
>> modification were my top priority.
>> Up to now things are at least working.
>>
>> ======================================================
>>
>> bye
>> Nicola
>>
>>
>>
>>
>>
>>
>>
>> On 10/21/21 14:53, Hernan Wilkinson wrote:
>>> Hi Nicola,
>>> I see your point regarding the functionality of upTo:, but
>>> you can easily overcome that using #peekBack. Using you
>>> example:
>>> -----
>>> s _ 'hello-1Ahello-2Ahel'.
>>> '/tmp/test.txt' asFileEntry fileContents: s.
>>>
>>> st1 _ '/tmp/test.txt' asFileEntry readStream .
>>>
>>> st1 upTo: $A. " 'hello-1' "
>>> st1 upTo: $A. " 'hello-2' "
>>> st1 upTo: $A. " 'hel' "
>>> (st1 atEnd and: [ st1 peekBack ~= $A ]) ifTrue: [ self
>>> error: 'End of file without delimiter ].
>>> ------
>>> Regarding my concern of adding this functionality to Cuis,
>>> we are trying to have a compact set of classes and methods
>>> to reduce complexity (or at least not increase it) and help
>>> newcomers to understand it and oldies to remember it :-) .
>>> We are also trying to add more and more tests because it is
>>> the only way to keep a system from becoming a legacy one and
>>> to reduce the fear it produces to change something.
>>> The strictUpTo:startPos: you are sending is almost a copy
>>> of the upTo: method, with a few lines changed. Even though
>>> the functionality makes sense (although right now you are
>>> the only one needing it and as I said, you can use peekBack
>>> to overcome it), adding that method adds repeated code which
>>> in the long term makes it more difficult to understand and
>>> maintain, even more because it does not have tests.
>>> So I hope you understand that as maintainers of Cuis, we
>>> want to be loyal to the goals I mentioned before and keep
>>> Cuis as clean and simple as possible. If you can refactor
>>> what you sent to avoid having repeated code with #upTo: and
>>> add tests that verify the functionality of both methods
>>> (strictUpTo: and upTo:), that will make our task easier and
>>> meet the goals we have. If you think this does not make
>>> sense to you, or you do not have the time to do it, it is
>>> completely understandable and in that case I suggest for you
>>> to have it as an extension of the StandardFileStream class
>>> or just use the peekBack message as I showed.
>>> I hope you understand my concern and agree with me. If not,
>>> please let me know.
>>>
>>> Cheers!
>>> Hernan.
>>>
>>>
>>> On Tue, Oct 19, 2021 at 10:32 AM Nicola Mingotti
>>> <nmingotti at gmail.com> wrote:
>>>
>>>
>>> Hi Hernan,
>>>
>>> In all frankness, in I would wipe out the old 'upTo'
>>> because its behavior is a bit "wild".
>>>
>>> On the other side, I understand it may create problems
>>> in retro-compatibility, that is why for
>>> the moment i propose to add a new method which behaves a
>>> bit better.
>>>
>>> I hope this example explains the problem:
>>> -------------------------------------------------------
>>> s _ 'hello-1Ahello-2Ahel'.
>>> '/tmp/test.txt' asFileEntry fileContents: s.
>>>
>>> st1 _ '/tmp/test.txt' asFileEntry readStream .
>>>
>>> st1 upTo: $A. " 'hello-1' "
>>> st1 upTo: $A. " 'hello-2' "
>>> st1 upTo: $A. " 'hel' " "(*)"
>>> ------------------------------------------------------
>>> (*) You can't establish in any way if you actually found
>>> an "A" terminated block or just hit the end of file
>>> (*) If you hit the end of file you eat an incomplete
>>> record, this is another problem, maybe another process
>>> was going to end writing that record but you will never
>>> know.
>>>
>>> Maybe there is another method around that performs
>>> similarly to 'strictUpTp', if there is I did not find
>>> it, sorry.
>>>
>>> IMHO, In a scale of importance from 0 to 10, this
>>> method, for a programmer, >= 8.
>>> I would definitely not put it into an external package,
>>> too much fundamental.
>>>
>>> bye
>>> Nicola
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On 10/19/21 14:44, Hernan Wilkinson wrote:
>>>> Hi Nicola!
>>>> I was wondering, why are you suggesting adding them to
>>>> the base? Is it not enough to implement them as an
>>>> extension in your package?
>>>> Also, I think that any new functionality should come
>>>> with its corresponding tests to help the maintenance
>>>> and understanding of the functionality.
>>>>
>>>> Cheers!
>>>> Hernan.
>>>>
>>>>
>>>> On Tue, Oct 19, 2021 at 7:04 AM Nicola Mingotti via
>>>> Cuis-dev <cuis-dev at lists.cuis.st> wrote:
>>>>
>>>> Hi Juan, guys,
>>>>
>>>> I would like to add to Cuis the 2 methods i attach
>>>> here. One is a helper method.
>>>>
>>>> -----------
>>>> StandardFileStream strictUpTo: delim.
>>>> -----------
>>>>
>>>> Differently from 'upTo: delim' this method:
>>>> 1. Does not return stuff if it does not find 'delim'.
>>>> 2. Does not upgrade the position on the stream if
>>>> does not find 'delim'.
>>>> 3. If it finds 'delim' returns a chunk that
>>>> includes it.
>>>>
>>>> I am parsing log files at the moment, this is very
>>>> much useful.
>>>>
>>>> NOTE. Up to now I tested only on small files.
>>>>
>>>> bye
>>>> Nicola
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Cuis-dev mailing list
>>>> Cuis-dev at lists.cuis.st
>>>> https://lists.cuis.st/mailman/listinfo/cuis-dev
>>>>
>>>>
>>>>
>>>> --
>>>> <https://10pines.com/>
>>>>
>>>>
>>>> Hernán Wilkinson
>>>>
>>>>
>>>> Software Developer & Coach
>>>>
>>>> Alem 896, Floor 6, Buenos Aires, Argentina
>>>>
>>>> +54 11 6091 3125
>>>>
>>>> @HernanWilkinson
>>>>
>>>
>>>
>>>
>>> --
>>> <https://10pines.com/>
>>>
>>>
>>> Hernán Wilkinson
>>>
>>>
>>> Software Developer & Coach
>>>
>>> Alem 896, Floor 6, Buenos Aires, Argentina
>>>
>>> +54 11 6091 3125
>>>
>>> @HernanWilkinson
>>>
>>
>>
>>
>> --
>> <https://10pines.com/>
>>
>>
>> Hernán Wilkinson
>>
>>
>> Software Developer & Coach
>>
>> Alem 896, Floor 6, Buenos Aires, Argentina
>>
>> +54 11 6091 3125
>>
>> @HernanWilkinson
>>
>
>
>
> --
> <https://10pines.com/>
>
>
> Hernán Wilkinson
>
>
> Software Developer & Coach
>
> Alem 896, Floor 6, Buenos Aires, Argentina
>
> +54 11 6091 3125
>
> @HernanWilkinson
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cuis.st/mailman/archives/cuis-dev/attachments/20211022/fbf655da/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 4930-CuisCore-NicolaMingotti-2021Oct22-12h07m-NM.001.cs.st
Type: application/vnd.sailingtracker.track
Size: 7147 bytes
Desc: not available
URL: <http://lists.cuis.st/mailman/archives/cuis-dev/attachments/20211022/fbf655da/attachment-0001.bin>
More information about the Cuis-dev
mailing list