<div dir="ltr">Hi all!<div> great work guys!! Nice tests too!! </div><div> Can I suggest changing the name of the tests to be more declarative? I think that will help a lot.</div><div><br></div><div> Nicola, about 'delimiterIsTerminator', that is a statement, not a question so Juan's selection on that keyword is correct from my perspective.</div><div><br></div><div>Cheers!</div><div>Hernan</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Oct 25, 2021 at 1:09 PM Nicola Mingotti <<a href="mailto:nmingotti@gmail.com">nmingotti@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<br>
Hi guys,<br>
<br>
Juan, I saw your changes, i think i understood. <br>
<br>
My only perplexity is about the parameter slot:
'delimiterIsTerminator', i guess<br>
if it has to be a question it should be 'isDelimiterTerminator' or
'isDelimiterATerminator' .<br>
<br>
I am not native English so my opinion on this weights as a feather.
<br>
<br>
bye<br>
Nicola<br>
<br>
<br>
<br>
<br>
<div>On 10/25/21 16:39, Juan Vuletich wrote:<br>
</div>
<blockquote type="cite">
Hi Nicola, Hernán,<br>
<br>
This is my take. I tried to be explicit and clear with
'terminator' vs. 'separator', and also added the other two
implementors of #upTo: in the Stream hierarchy. Slightly tweaked
your tests, and used them almost verbatim for the other two
implementors.<br>
<br>
Please review.<br>
<br>
Thanks,<br>
<br>
On 10/25/2021 7:35 AM, Nicola Mingotti via Cuis-dev wrote:
<blockquote type="cite">
<br>
Hi Juan,<br>
<br>
1. I corrected the bug you found, added other test cases and
made them symmetric<br>
between 'upTo' and 'upToStrict'. There are 2 files attached, one
for tests one to collect changes to System-Files.<br>
<br>
2. about names, 'terminator', 'separator', i see your point. I
am open to any<br>
naming scheme. The motivation that pushes me to ask this
enhancement<br>
of 'upTo' is totally based on log parsing. So, It wouldn't be
inappropriate also to name the<br>
boolean parameter something like "logReaderMode". It would be
long, but easy to detect<br>
for people involved in this kind of business. I don't dislike
also "strict" to be honest. <br>
<br>
<br>
bye<br>
Nicola<br>
<br>
<br>
<br>
<div>On 10/24/21 18:14, Juan Vuletich
wrote:<br>
</div>
<blockquote type="cite">
Hi Nicola,<br>
<br>
On 10/23/2021 6:56 PM, Nicola Mingotti via Cuis-dev wrote:
<blockquote type="cite">
<br>
Hi Juan,<br>
<br>
At the best of my current undestanding I can provide this:<br>
<br>
1. Fileout for tests in BaseImageTests<br>
</blockquote>
<br>
Much better!<br>
<br>
<blockquote type="cite"> 2. A few fileout for new methods and method
names<br>
</blockquote>
<br>
I still think the focus should be on terminator vs. separator,
especially on method and argument names. See <a href="https://en.wikipedia.org/wiki/Newline" target="_blank">https://en.wikipedia.org/wiki/Newline</a>
:<br>
<br>
"Interpretation<br>
Two ways to view newlines, both of which are self-consistent,
are that newlines either separate lines or that they terminate
lines. If a newline is considered a separator, there will be
no newline after the last line of a file. Some programs have
problems processing the last line of a file if it is not
terminated by a newline. On the other hand, programs that
expect newline to be used as a separator will interpret a
final newline as starting a new (empty) line. Conversely, if a
newline is considered a terminator, all text lines including
the last are expected to be terminated by a newline. If the
final character sequence in a text file is not a newline, the
final line of the file may be considered to be an improper or
incomplete text line, or the file may be considered to be
improperly truncated. "<br>
<br>
<blockquote type="cite"> 3. I could not replicate the bug you say, I did
not understand well maybe. if you could send me <br>
a fail example it would be helpful.<br>
<br>
</blockquote>
<br>
Sure. The following test fails even after fixing the obvious
bug:<br>
<br>
testUpToStrict3<br>
| path fs read |<br>
path := 'test-{1}.txt' format: {(Float pi * 10e10) floor.
} .<br>
path asFileEntry fileContents: ((1 to: 100) inject: ''
into: [ :prev :each | prev, 'A lot of stuff, needs over 2000
chars! ']). <br>
fs := path asFileEntry readStream.<br>
read := fs upTo: $X strict: true.<br>
self assert: (read = nil).<br>
fs close.<br>
<br>
<blockquote type="cite"> <br>
bye<br>
Nicola<br>
<br>
</blockquote>
<br>
Cheers,<br>
<br>
<blockquote type="cite"> <br>
<div>On 10/23/21 02:26, Nicola
Mingotti wrote:<br>
</div>
<blockquote type="cite">
<br>
Hi Juan, let me a bit of time to read your references, I
thought what I sent were test methods,<br>
clearly i miss part of the story.<br>
<br>
There shouldn't be any concatenation of nil and for God
sake NO partial records. <br>
This is what I wanted to avoid, apologies. <br>
<br>
Tomorrow i will probably be out for the Linux day, i will
update when possible.<br>
<br>
<br>
bye<br>
Nicola<br>
<br>
<br>
<br>
<br>
<div>On 10/23/21 01:20, Juan
Vuletich wrote:<br>
</div>
<blockquote type="cite">
Hi Folks,<br>
<br>
The main point here is not "strict vs. legacy",
"logically correct vs incorrect" or anything like that
at all.<br>
<br>
The point is "separator vs. terminator", and how using a
terminator instead of a separator allows processing
files while they are still being written to. (And this
has really no relation with running on a server or any
other kind of machine.)<br>
<br>
Besides, Nicola, your code has a bug when recurring on
terminator: it will answer the previous partial last
record concatenated with nil.<br>
<br>
Finally, please take a look at TestCase,SUnit and
<a href="http://BaseImageTests.pck.st" target="_blank">BaseImageTests.pck.st</a> to see what we mean by a "test".<br>
<br>
Thanks,<br>
<br>
On 10/22/2021 12:18 PM, Nicola Mingotti via Cuis-dev
wrote:
<blockquote type="cite">
<br>
Hi Hernan, <br>
<br>
We will have opportunity to work together on larger
problems, this is too small.<br>
It would take more time to talk than to do things ;)<br>
<br>
I have a proposed version. I rewrote the methods.
wrote the test. I kept a good part<br>
of the original code which may have evolved for
efficiency over time.<br>
<br>
upToLegacy method can of course be eliminated. it is
there only for reference. <br>
<br>
upTo: XXX --- now calls ---> upTo: XXX strict:
false <br>
<br>
upTo: XXX strict: XXX ------ is recursive, it needs an
extra helper method to remember a parameter (Scheme
recursion style) ----> upTo: XXX strict: XXX
posMemo: xxxx<br>
<br>
See attached fileout<br>
<br>
<br>
bye<br>
Nicola<br>
<br>
<br>
<br>
<br>
<br>
<div>On 10/21/21 19:49, Hernan
Wilkinson wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">ok, let me know. I wish we could do
it together but my agenda (and I guess yours) is
almost always full...</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Thu, Oct 21,
2021 at 2:32 PM Nicola Mingotti <<a href="mailto:nmingotti@gmail.com" target="_blank">nmingotti@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div> <br>
Hi Hernan,<br>
<br>
ok, let me try, it is too many days i am
talking about it.<br>
<br>
I will let you know soon<br>
<br>
bye<br>
Nicola<br>
<br>
<br>
<br>
<br>
<div>On 10/21/21 19:02, Hernan Wilkinson
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi Nicolas,
<div> if you could refactor upTo: to use
the same code as strictUpTo: and write
the tests to check that everything works
as expected, that would be great!</div>
<div> I would not use the names of the
Linux stdlib for those messages nor the
C functions, it is not necessary...</div>
<div> If you do not have the time to do
it, I can give it a try if you wish.</div>
<div><br>
</div>
<div>Cheers!</div>
<div>Hernan.<br>
<div> </div>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Thu,
Oct 21, 2021 at 12:47 PM Nicola Mingotti
<<a href="mailto:nmingotti@gmail.com" target="_blank">nmingotti@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div> <br>
Hi Hernan,<br>
<br>
. forget the code and test. I can
rewrite it from scratch with test. I
actually changed<br>
existing code for "politeness" ;) <br>
<br>
. for me it is very important to have
this matter fixed, well and for the
future.<br>
It is not good to have standard lib
functionality disseminated in my
application packages.<br>
<br>
. since I found Linux stdlib has a
function to do well what i want i will
use that name(s)<br>
to avoid confusion and recycle already
existing function names. "getline" and
"getdelim".<br>
<br>
. if you really dislike this functions
I can put them in OSProcess and maybe
<br>
just link the C version only for
Linux/BSD. So much I think they are
valuable in the server environment.<br>
<br>
. to fix this i need maybe 1-2 days.
If i need to link the C functions I
don't know, since I never tried.<br>
<br>
So, let me know, if you are not
against these functions I am open to
implement them well.<br>
<br>
<br>
===== Extra considerations whose
reading is secondary
==================<br>
<br>
. your fix was one step in the right
direction but not enough, you also
need to<br>
bring back the stream pointer to the
last existant $A. This is to say: too
complex.<br>
A good method must do all its chore,
not leave us back the dirty business
and special conditions.<br>
<br>
. I understand the concision, small
core etc. On the other side, i <br>
run Cuis on the servers. the most
important thing there is on servers
are files and<br>
sockets. You must read from there all
of the time. It must be easy and idiot
proof,<br>
rock solid and resistant to concurrent
processing as far as possible.<br>
<br>
. I see that Python and Ruby standard
library do it wrong, at bit better
than Cuis 'upTo' does.<br>
but still bad. They leave you the '\n'
at the end, but, if any process goes
on writing<br>
'f1.txt' Ruby and Python lost the half
backed record ! <br>
-------- Linux<br>
<span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)">$> printf
'line-1\nline-2\nline-TRAP' >
f1.txt<br>
# python<br>
$> </span></span><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)">python3.9 -c
"f=open('f1.txt','r');
print(f.readlines())" </span><br>
</span>=> </span></span><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)">['line-1\n',
'line-2\n', 'line-TRAP']</span><br>
# ruby <br>
$> </span></span></span><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)">ruby -e
"f=open('f1.txt','r'); puts
f.readlines().to_s; " </span><br>
=> ["line-1\n", "line-2\n",
"line-TRAP"]<br>
# both Python and Ruby ate the
half backed record ! bad !<br>
</span></span>---------------------------------------------------------<br>
</span></span><br>
. C and CommonLisp standard libraries
have a way to do it right:<br>
-) CL read-line. <a href="http://www.lispworks.com/documentation/HyperSpec/Body/f_rd_lin.htm#read-line" target="_blank">http://www.lispworks.com/documentation/HyperSpec/Body/f_rd_lin.htm#read-line</a>
<br>
-) C getline. <a href="https://man7.org/linux/man-pages/man3/getline.3.html" target="_blank">https://man7.org/linux/man-pages/man3/getline.3.html</a><br>
<br>
. I understand I am probably the only
one running Cuis in the server so I am
the first<br>
to step into a few particular
problems.<br>
<br>
. In my opinion Cuis in the Server can
be a good match, up to now i have 2
small<br>
company services working and a big one
project in continuous development. <br>
Time will tell. Sturdiness,
undertandability and ease of
modification were my top priority.<br>
Up to now things are at least working.<br>
<br>
======================================================<br>
<br>
bye<br>
Nicola<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<div>On 10/21/21 14:53, Hernan
Wilkinson wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi Nicola,
<div> I see your point regarding
the functionality of upTo:, but
you can easily overcome that
using #peekBack. Using you
example: </div>
<div>-----</div>
<div>s _ 'hello-1Ahello-2Ahel'.<br>
'/tmp/test.txt' asFileEntry
fileContents: s.<br>
<br>
st1 _ '/tmp/test.txt'
asFileEntry readStream .<br>
<br>
st1 upTo: $A. " 'hello-1' "</div>
<div>st1 upTo: $A. " 'hello-2' "<br>
st1 upTo: $A. " 'hel' " </div>
<div>(st1 atEnd and: [ st1
peekBack ~= $A ]) ifTrue: [ self
error: 'End of file without
delimiter ]. <br>
</div>
<div>------</div>
<div> </div>
<div> Regarding my concern of
adding this functionality to
Cuis, we are trying to have a
compact set of classes and
methods to reduce complexity (or
at least not increase it) and
help newcomers to understand it
and oldies to remember it :-) .
We are also trying to add more
and more tests because it is the
only way to keep a system from
becoming a legacy one and to
reduce the fear it produces to
change something.</div>
<div> The strictUpTo:startPos: you
are sending is almost a copy of
the upTo: method, with a few
lines changed. Even though the
functionality makes sense
(although right now you are the
only one needing it and as I
said, you can use peekBack to
overcome it), adding that method
adds repeated code which in the
long term makes it more
difficult to understand and
maintain, even more because it
does not have tests.</div>
<div> So I hope you understand
that as maintainers of Cuis, we
want to be loyal to the goals I
mentioned before and keep Cuis
as clean and simple as possible.
If you can refactor what you
sent to avoid having repeated
code with #upTo: and add tests
that verify the functionality of
both methods (strictUpTo: and
upTo:), that will make our task
easier and meet the goals we
have. If you think this does not
make sense to you, or you do not
have the time to do it, it is
completely understandable and in
that case I suggest for you to
have it as an extension of the
StandardFileStream class or just
use the peekBack message as I
showed.</div>
<div> I hope you understand my
concern and agree with me. If
not, please let me know.</div>
<div><br>
</div>
<div>Cheers!</div>
<div>Hernan.</div>
<div><br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On
Tue, Oct 19, 2021 at 10:32 AM
Nicola Mingotti <<a href="mailto:nmingotti@gmail.com" target="_blank">nmingotti@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div> <br>
Hi Hernan,<br>
<br>
In all frankness, in I would
wipe out the old 'upTo'
because its behavior is a bit
"wild".<br>
<br>
On the other side, I
understand it may create
problems in
retro-compatibility, that is
why for<br>
the moment i propose to add a
new method which behaves a bit
better.<br>
<br>
I hope this example explains
the problem:<br>
-------------------------------------------------------<br>
s _ 'hello-1Ahello-2Ahel'.<br>
'/tmp/test.txt' asFileEntry
fileContents: s. <br>
<br>
st1 _ '/tmp/test.txt'
asFileEntry readStream . <br>
<br>
st1 upTo: $A. " 'hello-1' "<br>
st1 upTo: $A. " 'hello-2' "<br>
st1 upTo: $A. " 'hel'
" "(*)"<br>
------------------------------------------------------<br>
(*) You can't establish in any
way if you actually found an
"A" terminated block or just
hit the end of file<br>
(*) If you hit the end of file
you eat an incomplete record,
this is another problem, maybe
another process<br>
was going to end writing that
record but you will never
know. <br>
<br>
Maybe there is another method
around that performs similarly
to 'strictUpTp', if there is I
did not find it, sorry.<br>
<br>
IMHO, In a scale of importance
from 0 to 10, this method, for
a programmer, >= 8.<br>
I would definitely not put it
into an external package, too
much fundamental.<br>
<br>
bye<br>
Nicola<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<div>On 10/19/21 14:44, Hernan
Wilkinson wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi Nicola!<br>
<div> I was wondering, why
are you suggesting
adding them to the base?
Is it not enough to
implement them as an
extension in your
package? </div>
<div> Also, I think that
any new
functionality should
come with its
corresponding tests to
help the maintenance and
understanding of the
functionality.</div>
<div><br>
</div>
<div>Cheers!</div>
<div>Hernan.</div>
<div><br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On
Tue, Oct 19, 2021 at
7:04 AM Nicola Mingotti
via Cuis-dev <<a href="mailto:cuis-dev@lists.cuis.st" target="_blank">cuis-dev@lists.cuis.st</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div> <font size="4"><font face="monospace">Hi
Juan, guys,<br>
<br>
I would like to
add to Cuis the 2
methods i attach
here. One is a
helper method.<br>
<br>
-----------<br>
StandardFileStream
strictUpTo: delim.<br>
-----------<br>
<br>
Differently from
'upTo: delim' this
method:<br>
1. Does not return
stuff if it does
not find 'delim'.<br>
2. Does not
upgrade the
position on the
stream if does not
find 'delim'.<br>
3. If it finds
'delim' returns a
chunk that
includes it.<br>
<br>
I am parsing log
files at the
moment, this is
very much useful.<br>
<br>
NOTE. Up to now I
tested only on
small files.<br>
<br>
bye<br>
Nicola<br>
<br>
<br>
<br>
<br>
</font></font> </div>
-- <br>
Cuis-dev mailing list<br>
<a href="mailto:Cuis-dev@lists.cuis.st" target="_blank">Cuis-dev@lists.cuis.st</a><br>
<a href="https://lists.cuis.st/mailman/listinfo/cuis-dev" rel="noreferrer" target="_blank">https://lists.cuis.st/mailman/listinfo/cuis-dev</a><br>
</blockquote>
</div>
<br>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</blockquote>
<br>
<pre cols="72">--
Juan Vuletich
<a href="http://www.cuis-smalltalk.org" target="_blank">www.cuis-smalltalk.org</a>
<a href="https://github.com/Cuis-Smalltalk/Cuis-Smalltalk-Dev" target="_blank">https://github.com/Cuis-Smalltalk/Cuis-Smalltalk-Dev</a>
<a href="https://github.com/jvuletich" target="_blank">https://github.com/jvuletich</a>
<a href="https://www.linkedin.com/in/juan-vuletich-75611b3" target="_blank">https://www.linkedin.com/in/juan-vuletich-75611b3</a>
@JuanVuletich</pre>
</blockquote>
<br>
</blockquote>
<br>
</blockquote>
<br>
<br>
<pre cols="72">--
Juan Vuletich
<a href="http://www.cuis-smalltalk.org" target="_blank">www.cuis-smalltalk.org</a>
<a href="https://github.com/Cuis-Smalltalk/Cuis-Smalltalk-Dev" target="_blank">https://github.com/Cuis-Smalltalk/Cuis-Smalltalk-Dev</a>
<a href="https://github.com/jvuletich" target="_blank">https://github.com/jvuletich</a>
<a href="https://www.linkedin.com/in/juan-vuletich-75611b3" target="_blank">https://www.linkedin.com/in/juan-vuletich-75611b3</a>
@JuanVuletich</pre>
</blockquote>
<br>
</blockquote>
<br>
<br>
<pre cols="72">--
Juan Vuletich
<a href="http://www.cuis-smalltalk.org" target="_blank">www.cuis-smalltalk.org</a>
<a href="https://github.com/Cuis-Smalltalk/Cuis-Smalltalk-Dev" target="_blank">https://github.com/Cuis-Smalltalk/Cuis-Smalltalk-Dev</a>
<a href="https://github.com/jvuletich" target="_blank">https://github.com/jvuletich</a>
<a href="https://www.linkedin.com/in/juan-vuletich-75611b3" target="_blank">https://www.linkedin.com/in/juan-vuletich-75611b3</a>
@JuanVuletich</pre>
</blockquote>
<br>
</div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><span style="font-size:xx-small;border-collapse:collapse"><div style="font-size:small"><a href="https://10pines.com/" style="font-family:Roboto,Helvetica,Arial,sans-serif;font-size:medium" target="_blank"><img width="108" src="https://10pines.github.io/email-signature/10pines-firma@2x.png" style="margin-bottom: 0.5em;"></a><span style="color:rgb(0,0,0);font-family:Roboto,Helvetica,Arial,sans-serif;font-size:medium"></span><h1 style="margin:0px;font-size:14px">Hernán Wilkinson</h1><h2 style="margin:0px 0px 1em;font-size:14px;color:rgb(100,100,100)">Software Developer & Coach</h2><p style="margin:0px;color:rgb(100,100,100);font-size:12px">Alem 896, Floor 6, Buenos Aires, Argentina</p><p style="margin:0px;color:rgb(100,100,100);font-size:12px">+54 11 6091 3125</p><p style="margin:0px;color:rgb(100,100,100);font-size:12px">@HernanWilkinson</p></div></span></div></div>