<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
<title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
Hi Nicola, Hernán,<br>
<br>
This is my take. I tried to be explicit and clear with 'terminator'
vs. 'separator', and also added the other two implementors of #upTo:
in the Stream hierarchy. Slightly tweaked your tests, and used them
almost verbatim for the other two implementors.<br>
<br>
Please review.<br>
<br>
Thanks,<br>
<br>
On 10/25/2021 7:35 AM, Nicola Mingotti via Cuis-dev wrote:
<blockquote
cite="mid:e3a163e0-b77a-4d07-1586-bba34a25eca6@gmail.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<br>
Hi Juan,<br>
<br>
1. I corrected the bug you found, added other test cases and made
them symmetric<br>
between 'upTo' and 'upToStrict'. There are 2 files attached, one
for tests one to collect changes to System-Files.<br>
<br>
2. about names, 'terminator', 'separator', i see your point. I am
open to any<br>
naming scheme. The motivation that pushes me to ask this
enhancement<br>
of 'upTo' is totally based on log parsing. So, It wouldn't be
inappropriate also to name the<br>
boolean parameter something like "logReaderMode". It would be
long, but easy to detect<br>
for people involved in this kind of business. I don't dislike also
"strict" to be honest. <br>
<br>
<br>
bye<br>
Nicola<br>
<br>
<br>
<br>
<div class="moz-cite-prefix">On 10/24/21 18:14, Juan Vuletich
wrote:<br>
</div>
<blockquote type="cite" cite="mid:6175864D.6060106@jvuletich.org">
<meta content="text/html; charset=UTF-8"
http-equiv="Content-Type">
Hi Nicola,<br>
<br>
On 10/23/2021 6:56 PM, Nicola Mingotti via Cuis-dev wrote:
<blockquote
cite="mid:09116dd9-b71a-aefb-8d8b-bd0ebbbb14fd@gmail.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=UTF-8">
<br>
Hi Juan,<br>
<br>
At the best of my current undestanding I can provide this:<br>
<br>
1. Fileout for tests in BaseImageTests<br>
</blockquote>
<br>
Much better!<br>
<br>
<blockquote
cite="mid:09116dd9-b71a-aefb-8d8b-bd0ebbbb14fd@gmail.com"
type="cite"> 2. A few fileout for new methods and method names<br>
</blockquote>
<br>
I still think the focus should be on terminator vs. separator,
especially on method and argument names. See <a
class="moz-txt-link-freetext"
href="https://en.wikipedia.org/wiki/Newline"
moz-do-not-send="true">https://en.wikipedia.org/wiki/Newline</a>
:<br>
<br>
"Interpretation<br>
Two ways to view newlines, both of which are self-consistent,
are that newlines either separate lines or that they terminate
lines. If a newline is considered a separator, there will be no
newline after the last line of a file. Some programs have
problems processing the last line of a file if it is not
terminated by a newline. On the other hand, programs that expect
newline to be used as a separator will interpret a final newline
as starting a new (empty) line. Conversely, if a newline is
considered a terminator, all text lines including the last are
expected to be terminated by a newline. If the final character
sequence in a text file is not a newline, the final line of the
file may be considered to be an improper or incomplete text
line, or the file may be considered to be improperly truncated.
"<br>
<br>
<blockquote
cite="mid:09116dd9-b71a-aefb-8d8b-bd0ebbbb14fd@gmail.com"
type="cite"> 3. I could not replicate the bug you say, I did
not understand well maybe. if you could send me <br>
a fail example it would be helpful.<br>
<br>
</blockquote>
<br>
Sure. The following test fails even after fixing the obvious
bug:<br>
<br>
testUpToStrict3<br>
| path fs read |<br>
path := 'test-{1}.txt' format: {(Float pi * 10e10) floor. }
.<br>
path asFileEntry fileContents: ((1 to: 100) inject: '' into:
[ :prev :each | prev, 'A lot of stuff, needs over 2000 chars!
']). <br>
fs := path asFileEntry readStream.<br>
read := fs upTo: $X strict: true.<br>
self assert: (read = nil).<br>
fs close.<br>
<br>
<blockquote
cite="mid:09116dd9-b71a-aefb-8d8b-bd0ebbbb14fd@gmail.com"
type="cite"> <br>
bye<br>
Nicola<br>
<br>
</blockquote>
<br>
Cheers,<br>
<br>
<blockquote
cite="mid:09116dd9-b71a-aefb-8d8b-bd0ebbbb14fd@gmail.com"
type="cite"> <br>
<div class="moz-cite-prefix">On 10/23/21 02:26, Nicola
Mingotti wrote:<br>
</div>
<blockquote type="cite"
cite="mid:5f2ef2d5-71ed-ddd8-962d-5ad63a8301ed@gmail.com">
<meta http-equiv="Content-Type" content="text/html;
charset=UTF-8">
<br>
Hi Juan, let me a bit of time to read your references, I
thought what I sent were test methods,<br>
clearly i miss part of the story.<br>
<br>
There shouldn't be any concatenation of nil and for God sake
NO partial records. <br>
This is what I wanted to avoid, apologies. <br>
<br>
Tomorrow i will probably be out for the Linux day, i will
update when possible.<br>
<br>
<br>
bye<br>
Nicola<br>
<br>
<br>
<br>
<br>
<div class="moz-cite-prefix">On 10/23/21 01:20, Juan
Vuletich wrote:<br>
</div>
<blockquote type="cite"
cite="mid:61734724.5010009@jvuletich.org">
<meta content="text/html; charset=UTF-8"
http-equiv="Content-Type">
Hi Folks,<br>
<br>
The main point here is not "strict vs. legacy", "logically
correct vs incorrect" or anything like that at all.<br>
<br>
The point is "separator vs. terminator", and how using a
terminator instead of a separator allows processing files
while they are still being written to. (And this has
really no relation with running on a server or any other
kind of machine.)<br>
<br>
Besides, Nicola, your code has a bug when recurring on
terminator: it will answer the previous partial last
record concatenated with nil.<br>
<br>
Finally, please take a look at TestCase,SUnit and
BaseImageTests.pck.st to see what we mean by a "test".<br>
<br>
Thanks,<br>
<br>
On 10/22/2021 12:18 PM, Nicola Mingotti via Cuis-dev
wrote:
<blockquote
cite="mid:79ec566d-c417-6eea-ec5c-01965d0cf93e@gmail.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=UTF-8">
<br>
Hi Hernan, <br>
<br>
We will have opportunity to work together on larger
problems, this is too small.<br>
It would take more time to talk than to do things ;)<br>
<br>
I have a proposed version. I rewrote the methods. wrote
the test. I kept a good part<br>
of the original code which may have evolved for
efficiency over time.<br>
<br>
upToLegacy method can of course be eliminated. it is
there only for reference. <br>
<br>
upTo: XXX --- now calls ---> upTo: XXX strict: false
<br>
<br>
upTo: XXX strict: XXX ------ is recursive, it needs an
extra helper method to remember a parameter (Scheme
recursion style) ----> upTo: XXX strict: XXX
posMemo: xxxx<br>
<br>
See attached fileout<br>
<br>
<br>
bye<br>
Nicola<br>
<br>
<br>
<br>
<br>
<br>
<div class="moz-cite-prefix">On 10/21/21 19:49, Hernan
Wilkinson wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAJAbP8jpOucdF34SNqQFVavbKdozS294rbpLsT4pXO9165hdcg@mail.gmail.com">
<meta http-equiv="content-type" content="text/html;
charset=UTF-8">
<div dir="ltr">ok, let me know. I wish we could do it
together but my agenda (and I guess yours) is almost
always full...</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Thu, Oct 21,
2021 at 2:32 PM Nicola Mingotti <<a
href="mailto:nmingotti@gmail.com"
moz-do-not-send="true"
class="moz-txt-link-freetext">nmingotti@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin: 0px
0px 0px 0.8ex; border-left: 1px solid rgb(204,
204, 204); padding-left: 1ex;">
<div> <br>
Hi Hernan,<br>
<br>
ok, let me try, it is too many days i am talking
about it.<br>
<br>
I will let you know soon<br>
<br>
bye<br>
Nicola<br>
<br>
<br>
<br>
<br>
<div>On 10/21/21 19:02, Hernan Wilkinson wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi Nicolas,
<div> if you could refactor upTo: to use the
same code as strictUpTo: and write the
tests to check that everything works as
expected, that would be great!</div>
<div> I would not use the names of the Linux
stdlib for those messages nor the C
functions, it is not necessary...</div>
<div> If you do not have the time to do it,
I can give it a try if you wish.</div>
<div><br>
</div>
<div>Cheers!</div>
<div>Hernan.<br>
<div> </div>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Thu,
Oct 21, 2021 at 12:47 PM Nicola Mingotti
<<a href="mailto:nmingotti@gmail.com"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">nmingotti@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote"
style="margin: 0px 0px 0px 0.8ex;
border-left: 1px solid rgb(204, 204, 204);
padding-left: 1ex;">
<div> <br>
Hi Hernan,<br>
<br>
. forget the code and test. I can
rewrite it from scratch with test. I
actually changed<br>
existing code for "politeness" ;) <br>
<br>
. for me it is very important to have
this matter fixed, well and for the
future.<br>
It is not good to have standard lib
functionality disseminated in my
application packages.<br>
<br>
. since I found Linux stdlib has a
function to do well what i want i will
use that name(s)<br>
to avoid confusion and recycle already
existing function names. "getline" and
"getdelim".<br>
<br>
. if you really dislike this functions I
can put them in OSProcess and maybe <br>
just link the C version only for
Linux/BSD. So much I think they are
valuable in the server environment.<br>
<br>
. to fix this i need maybe 1-2 days. If
i need to link the C functions I don't
know, since I never tried.<br>
<br>
So, let me know, if you are not against
these functions I am open to implement
them well.<br>
<br>
<br>
===== Extra considerations whose reading
is secondary ==================<br>
<br>
. your fix was one step in the right
direction but not enough, you also need
to<br>
bring back the stream pointer to the
last existant $A. This is to say: too
complex.<br>
A good method must do all its chore, not
leave us back the dirty business and
special conditions.<br>
<br>
. I understand the concision, small core
etc. On the other side, i <br>
run Cuis on the servers. the most
important thing there is on servers are
files and<br>
sockets. You must read from there all of
the time. It must be easy and idiot
proof,<br>
rock solid and resistant to concurrent
processing as far as possible.<br>
<br>
. I see that Python and Ruby standard
library do it wrong, at bit better than
Cuis 'upTo' does.<br>
but still bad. They leave you the '\n'
at the end, but, if any process goes on
writing<br>
'f1.txt' Ruby and Python lost the half
backed record ! <br>
-------- Linux<br>
<span style="font-family: monospace;"><span
style="color: rgb(0, 0, 0);
background-color: rgb(255, 255,
255);">$> printf
'line-1\nline-2\nline-TRAP' >
f1.txt<br>
# python<br>
$> </span></span><span
style="font-family: monospace;"><span
style="color: rgb(0, 0, 0);
background-color: rgb(255, 255,
255);"><span style="font-family:
monospace;"><span style="color:
rgb(0, 0, 0); background-color:
rgb(255, 255, 255);">python3.9
-c "f=open('f1.txt','r');
print(f.readlines())" </span><br>
</span>=> </span></span><span
style="font-family: monospace;"><span
style="color: rgb(0, 0, 0);
background-color: rgb(255, 255,
255);"><span style="font-family:
monospace;"><span style="color:
rgb(0, 0, 0); background-color:
rgb(255, 255, 255);">['line-1\n',
'line-2\n', 'line-TRAP']</span><br>
# ruby <br>
$> </span></span></span><span
style="font-family: monospace;"><span
style="color: rgb(0, 0, 0);
background-color: rgb(255, 255,
255);"><span style="font-family:
monospace;"><span
style="font-family: monospace;"><span
style="color: rgb(0, 0, 0);
background-color: rgb(255,
255, 255);">ruby -e
"f=open('f1.txt','r'); puts
f.readlines().to_s; " </span><br>
=> ["line-1\n", "line-2\n",
"line-TRAP"]<br>
# both Python and Ruby ate the
half backed record ! bad !<br>
</span></span>---------------------------------------------------------<br>
</span></span><br>
. C and CommonLisp standard libraries
have a way to do it right:<br>
-) CL read-line. <a
href="http://www.lispworks.com/documentation/HyperSpec/Body/f_rd_lin.htm#read-line"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">http://www.lispworks.com/documentation/HyperSpec/Body/f_rd_lin.htm#read-line</a>
<br>
-) C getline. <a
href="https://man7.org/linux/man-pages/man3/getline.3.html"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">https://man7.org/linux/man-pages/man3/getline.3.html</a><br>
<br>
. I understand I am probably the only
one running Cuis in the server so I am
the first<br>
to step into a few particular problems.<br>
<br>
. In my opinion Cuis in the Server can
be a good match, up to now i have 2
small<br>
company services working and a big one
project in continuous development. <br>
Time will tell. Sturdiness,
undertandability and ease of
modification were my top priority.<br>
Up to now things are at least working.<br>
<br>
======================================================<br>
<br>
bye<br>
Nicola<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<div>On 10/21/21 14:53, Hernan Wilkinson
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi Nicola,
<div> I see your point regarding the
functionality of upTo:, but you
can easily overcome that using
#peekBack. Using you example: </div>
<div>-----</div>
<div>s _ 'hello-1Ahello-2Ahel'.<br>
'/tmp/test.txt' asFileEntry
fileContents: s.<br>
<br>
st1 _ '/tmp/test.txt' asFileEntry
readStream .<br>
<br>
st1 upTo: $A. " 'hello-1' "</div>
<div>st1 upTo: $A. " 'hello-2' "<br>
st1 upTo: $A. " 'hel' " </div>
<div>(st1 atEnd and: [ st1 peekBack
~= $A ]) ifTrue: [ self error:
'End of file without delimiter ].
<br>
</div>
<div>------</div>
<div> </div>
<div> Regarding my concern of adding
this functionality to Cuis, we are
trying to have a compact set of
classes and methods to reduce
complexity (or at least not
increase it) and help newcomers to
understand it and oldies to
remember it :-) . We are also
trying to add more and more tests
because it is the only way to keep
a system from becoming a legacy
one and to reduce the fear it
produces to change something.</div>
<div> The strictUpTo:startPos: you
are sending is almost a copy of
the upTo: method, with a few lines
changed. Even though the
functionality makes sense
(although right now you are the
only one needing it and as I said,
you can use peekBack to overcome
it), adding that method adds
repeated code which in the long
term makes it more difficult to
understand and maintain, even more
because it does not have tests.</div>
<div> So I hope you understand that
as maintainers of Cuis, we want to
be loyal to the goals I mentioned
before and keep Cuis as clean and
simple as possible. If you can
refactor what you sent to avoid
having repeated code with #upTo:
and add tests that verify the
functionality of both methods
(strictUpTo: and upTo:), that will
make our task easier and meet the
goals we have. If you think this
does not make sense to you, or you
do not have the time to do it, it
is completely understandable and
in that case I suggest for you to
have it as an extension of the
StandardFileStream class or just
use the peekBack message as I
showed.</div>
<div> I hope you understand my
concern and agree with me. If not,
please let me know.</div>
<div><br>
</div>
<div>Cheers!</div>
<div>Hernan.</div>
<div><br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On
Tue, Oct 19, 2021 at 10:32 AM
Nicola Mingotti <<a
href="mailto:nmingotti@gmail.com"
target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">nmingotti@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote"
style="margin: 0px 0px 0px 0.8ex;
border-left: 1px solid rgb(204,
204, 204); padding-left: 1ex;">
<div> <br>
Hi Hernan,<br>
<br>
In all frankness, in I would
wipe out the old 'upTo' because
its behavior is a bit "wild".<br>
<br>
On the other side, I understand
it may create problems in
retro-compatibility, that is why
for<br>
the moment i propose to add a
new method which behaves a bit
better.<br>
<br>
I hope this example explains the
problem:<br>
-------------------------------------------------------<br>
s _ 'hello-1Ahello-2Ahel'.<br>
'/tmp/test.txt' asFileEntry
fileContents: s. <br>
<br>
st1 _ '/tmp/test.txt'
asFileEntry readStream . <br>
<br>
st1 upTo: $A. " 'hello-1' "<br>
st1 upTo: $A. " 'hello-2' "<br>
st1 upTo: $A. " 'hel' "
"(*)"<br>
------------------------------------------------------<br>
(*) You can't establish in any
way if you actually found an "A"
terminated block or just hit the
end of file<br>
(*) If you hit the end of file
you eat an incomplete record,
this is another problem, maybe
another process<br>
was going to end writing that
record but you will never know.
<br>
<br>
Maybe there is another method
around that performs similarly
to 'strictUpTp', if there is I
did not find it, sorry.<br>
<br>
IMHO, In a scale of importance
from 0 to 10, this method, for a
programmer, >= 8.<br>
I would definitely not put it
into an external package, too
much fundamental.<br>
<br>
bye<br>
Nicola<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<div>On 10/19/21 14:44, Hernan
Wilkinson wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi Nicola!<br>
<div> I was wondering, why
are you suggesting adding
them to the base? Is it
not enough to implement
them as an extension in
your package? </div>
<div> Also, I think that any
new functionality should
come with its
corresponding tests to
help the maintenance and
understanding of the
functionality.</div>
<div><br>
</div>
<div>Cheers!</div>
<div>Hernan.</div>
<div><br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr"
class="gmail_attr">On Tue,
Oct 19, 2021 at 7:04 AM
Nicola Mingotti via
Cuis-dev <<a
href="mailto:cuis-dev@lists.cuis.st"
target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">cuis-dev@lists.cuis.st</a>>
wrote:<br>
</div>
<blockquote
class="gmail_quote"
style="margin: 0px 0px 0px
0.8ex; border-left: 1px
solid rgb(204, 204, 204);
padding-left: 1ex;">
<div> <font size="4"><font
face="monospace">Hi
Juan, guys,<br>
<br>
I would like to add
to Cuis the 2
methods i attach
here. One is a
helper method.<br>
<br>
-----------<br>
StandardFileStream
strictUpTo: delim.<br>
-----------<br>
<br>
Differently from
'upTo: delim' this
method:<br>
1. Does not return
stuff if it does not
find 'delim'.<br>
2. Does not upgrade
the position on the
stream if does not
find 'delim'.<br>
3. If it finds
'delim' returns a
chunk that includes
it.<br>
<br>
I am parsing log
files at the moment,
this is very much
useful.<br>
<br>
NOTE. Up to now I
tested only on small
files.<br>
<br>
bye<br>
Nicola<br>
<br>
<br>
<br>
<br>
</font></font> </div>
-- <br>
Cuis-dev mailing list<br>
<a
href="mailto:Cuis-dev@lists.cuis.st"
target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">Cuis-dev@lists.cuis.st</a><br>
<a
href="https://lists.cuis.st/mailman/listinfo/cuis-dev"
rel="noreferrer"
target="_blank"
moz-do-not-send="true"
class="moz-txt-link-freetext">https://lists.cuis.st/mailman/listinfo/cuis-dev</a><br>
</blockquote>
</div>
<br>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Juan Vuletich
<a class="moz-txt-link-abbreviated" href="http://www.cuis-smalltalk.org" moz-do-not-send="true">www.cuis-smalltalk.org</a>
<a class="moz-txt-link-freetext" href="https://github.com/Cuis-Smalltalk/Cuis-Smalltalk-Dev" moz-do-not-send="true">https://github.com/Cuis-Smalltalk/Cuis-Smalltalk-Dev</a>
<a class="moz-txt-link-freetext" href="https://github.com/jvuletich" moz-do-not-send="true">https://github.com/jvuletich</a>
<a class="moz-txt-link-freetext" href="https://www.linkedin.com/in/juan-vuletich-75611b3" moz-do-not-send="true">https://www.linkedin.com/in/juan-vuletich-75611b3</a>
@JuanVuletich</pre>
</blockquote>
<br>
</blockquote>
<br>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Juan Vuletich
<a class="moz-txt-link-abbreviated" href="http://www.cuis-smalltalk.org" moz-do-not-send="true">www.cuis-smalltalk.org</a>
<a class="moz-txt-link-freetext" href="https://github.com/Cuis-Smalltalk/Cuis-Smalltalk-Dev" moz-do-not-send="true">https://github.com/Cuis-Smalltalk/Cuis-Smalltalk-Dev</a>
<a class="moz-txt-link-freetext" href="https://github.com/jvuletich" moz-do-not-send="true">https://github.com/jvuletich</a>
<a class="moz-txt-link-freetext" href="https://www.linkedin.com/in/juan-vuletich-75611b3" moz-do-not-send="true">https://www.linkedin.com/in/juan-vuletich-75611b3</a>
@JuanVuletich</pre>
</blockquote>
<br>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Juan Vuletich
<a class="moz-txt-link-abbreviated" href="http://www.cuis-smalltalk.org">www.cuis-smalltalk.org</a>
<a class="moz-txt-link-freetext" href="https://github.com/Cuis-Smalltalk/Cuis-Smalltalk-Dev">https://github.com/Cuis-Smalltalk/Cuis-Smalltalk-Dev</a>
<a class="moz-txt-link-freetext" href="https://github.com/jvuletich">https://github.com/jvuletich</a>
<a class="moz-txt-link-freetext" href="https://www.linkedin.com/in/juan-vuletich-75611b3">https://www.linkedin.com/in/juan-vuletich-75611b3</a>
@JuanVuletich</pre>
</body>
</html>