| Author |
Message |
|
| jay |
 Posted: Mon May 17, 2010 9:24 am |
 |
|
|
User
Joined: 06 Sep 2006
Posts: 139
Location: Los Angeles, CA USA
|
Bjorn,
I pushed a new behaviour to stdlib called gen_stream which implements
a process-striped buffer for reading files, binaries or generated
streams as a serial set of data blocks. This is a complete rewrite
of a submission I made at the end of 2007 (see http://www.duomark.com/
erlang/proposals/gen_stream.html for the original background on the
project).
I created a fairly thorough gen_stream_SUITE.erl and official
documentation in the gen_stream.xml file. It is stable enough to
use, although I need to add some more tests to the SUITE (timeout,
hibernate, ignore and stop on initialization, code_change, and an
example behaviour that uses dets handles which must be maintained in
the module state. I also need to debug issues I am having on a
single core laptop with too many processes (not sure if this is the
implementation or a problem with the external behaviour module
example I am testing with).
The API is slightly changed from the original; the API for the
behaviour is more different from the original. I added the ability
to pass a transform fun or mfa to reformat each data block as it is
written to internal buffers. This allows newline delimited text to
be split efficiently, and allows for any arbitrary transformation of
the data type or format but only independently within each block of
data.
Will this module require an EEP?
You can retrieve the new code from my account:
git fetch git://github.com/jaynel/otp.git gen_stream
jay
________________________________________________________________
erlang-patches (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-patches-unsubscribe@erlang.org
Post received from mailinglist |
|
|
| Back to top |
  
|
| jay |
| Posted: Mon May 17, 2010 9:25 am |
|
|
|
User
Joined: 06 Sep 2006
Posts: 139
Location: Los Angeles, CA USA
|
Bjorn,
I pushed a new behaviour to stdlib called gen_stream which implements
a process-striped buffer for reading files, binaries or generated
streams as a serial set of data blocks. This is a complete rewrite
of a submission I made at the end of 2007 (see http://www.duomark.com/
erlang/proposals/gen_stream.html for the original background on the
project).
I created a fairly thorough gen_stream_SUITE.erl and official
documentation in the gen_stream.xml file. It is stable enough to
use, although I need to add some more tests to the SUITE (timeout,
hibernate, ignore and stop on initialization, code_change, and an
example behaviour that uses dets handles which must be maintained in
the module state. I also need to debug issues I am having on a
single core laptop with too many processes (not sure if this is the
implementation or a problem with the external behaviour module
example I am testing with).
The API is slightly changed from the original; the API for the
behaviour is more different from the original. I added the ability
to pass a transform fun or mfa to reformat each data block as it is
written to internal buffers. This allows newline delimited text to
be split efficiently, and allows for any arbitrary transformation of
the data type or format but only independently within each block of
data.
Will this module require an EEP?
You can retrieve the new code from my account:
git fetch git://github.com/jaynel/otp.git gen_stream
jay
________________________________________________________________
erlang-patches (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-patches-unsubscribe@erlang.org
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Mon May 17, 2010 1:33 pm |
|
|
|
Guest
|
On Mon, May 17, 2010 at 02:25:09AM -0700, Jay Nelson wrote:
> Bjorn,
Not this time 
>
> Will this module require an EEP?
I do not think so since it does not affect the language
nor the core modules. It may, however, need to be reviewed by
the OTP technical board. We would also like community feedback
on the behaviour of a new behaviour.
>
> You can retrieve the new code from my account:
>
> git fetch git://github.com/jaynel/otp.git gen_stream
Thanks. Included in 'pu' with whitespace fixes.
--
/ Raimo Niskanen, Erlang/OTP, Ericsson AB
________________________________________________________________
erlang-patches (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-patches-unsubscribe@erlang.org
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Mon May 17, 2010 1:33 pm |
|
|
|
Guest
|
On Mon, May 17, 2010 at 02:25:09AM -0700, Jay Nelson wrote:
> Bjorn,
Not this time
>
> Will this module require an EEP?
I do not think so since it does not affect the language
nor the core modules. It may, however, need to be reviewed by
the OTP technical board. We would also like community feedback
on the behaviour of a new behaviour.
>
> You can retrieve the new code from my account:
>
> git fetch git://github.com/jaynel/otp.git gen_stream
Thanks. Included in 'pu' with whitespace fixes.
--
/ Raimo Niskanen, Erlang/OTP, Ericsson AB
________________________________________________________________
erlang-patches (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-patches-unsubscribe@erlang.org
Post received from mailinglist |
|
|
| Back to top |
|
| jay |
| Posted: Mon May 17, 2010 4:28 pm |
|
|
|
User
Joined: 06 Sep 2006
Posts: 139
Location: Los Angeles, CA USA
|
On May 17, 2010, at 6:33 AM, Raimo Niskanen wrote:
>
> I do not think so since it does not affect the language
> nor the core modules. It may, however, need to be reviewed by
> the OTP technical board. We would also like community feedback
> on the behaviour of a new behaviour.
Please, absolutely. I definitely want feedback. I am sure that
there are many improvements possible that I have not considered.
>>
>> git fetch git://github.com/jaynel/otp.git gen_stream
>
> Thanks. Included in 'pu' with whitespace fixes.
How is this done? I did 'git diff --check' and didn't get
anything. How do you detect and fix whitespace issues?
jay
________________________________________________________________
erlang-patches (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-patches-unsubscribe@erlang.org
Post received from mailinglist |
|
|
| Back to top |
|
| jay |
| Posted: Mon May 17, 2010 4:29 pm |
|
|
|
User
Joined: 06 Sep 2006
Posts: 139
Location: Los Angeles, CA USA
|
On May 17, 2010, at 6:33 AM, Raimo Niskanen wrote:
>
> I do not think so since it does not affect the language
> nor the core modules. It may, however, need to be reviewed by
> the OTP technical board. We would also like community feedback
> on the behaviour of a new behaviour.
Please, absolutely. I definitely want feedback. I am sure that
there are many improvements possible that I have not considered.
>>
>> git fetch git://github.com/jaynel/otp.git gen_stream
>
> Thanks. Included in 'pu' with whitespace fixes.
How is this done? I did 'git diff --check' and didn't get
anything. How do you detect and fix whitespace issues?
jay
________________________________________________________________
erlang-patches (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-patches-unsubscribe@erlang.org
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Mon May 17, 2010 8:53 pm |
|
|
|
Guest
|
>> We would also like community feedback
>> on the behaviour of a new behaviour.
>
> Please, absolutely. I definitely want feedback. I am sure that there are
> many improvements possible that I have not considered.
I haven't tried the code, but like the idea of a stream behaviour/API.
How about including writable streams as well, similar to Node
(http://nodejs.org/api.html#streams-20)?
Tim
________________________________________________________________
erlang-patches (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-patches-unsubscribe@erlang.org
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Mon May 17, 2010 8:54 pm |
|
|
|
Guest
|
>> We would also like community feedback
>> on the behaviour of a new behaviour.
>
> Please, absolutely. I definitely want feedback. I am sure that there are
> many improvements possible that I have not considered.
I haven't tried the code, but like the idea of a stream behaviour/API.
How about including writable streams as well, similar to Node
(http://nodejs.org/api.html#streams-20)?
Tim
________________________________________________________________
erlang-patches (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-patches-unsubscribe@erlang.org
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Tue May 18, 2010 8:21 am |
|
|
|
Guest
|
On Mon, May 17, 2010 at 09:29:09AM -0700, Jay Nelson wrote:
>
> On May 17, 2010, at 6:33 AM, Raimo Niskanen wrote:
> >
> >I do not think so since it does not affect the language
> >nor the core modules. It may, however, need to be reviewed by
> >the OTP technical board. We would also like community feedback
> >on the behaviour of a new behaviour.
>
> Please, absolutely. I definitely want feedback. I am sure that
> there are many improvements possible that I have not considered.
>
> >>
> >>git fetch git://github.com/jaynel/otp.git gen_stream
> >
> >Thanks. Included in 'pu' with whitespace fixes.
>
> How is this done? I did 'git diff --check' and didn't get
> anything. How do you detect and fix whitespace issues?
There is a hook .git/hooks/pre-commit.sample
that you rename to .git/hooks/pre-commit
and then you can not commit any whitespace errors
(some other problems are checked too).
The argument --check to git diff checks for whitespace errors.
To fix I use git rebase --whitespace=fix
>
> jay
--
/ Raimo Niskanen, Erlang/OTP, Ericsson AB
________________________________________________________________
erlang-patches (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-patches-unsubscribe@erlang.org
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Tue May 18, 2010 8:22 am |
|
|
|
Guest
|
On Mon, May 17, 2010 at 09:29:09AM -0700, Jay Nelson wrote:
>
> On May 17, 2010, at 6:33 AM, Raimo Niskanen wrote:
> >
> >I do not think so since it does not affect the language
> >nor the core modules. It may, however, need to be reviewed by
> >the OTP technical board. We would also like community feedback
> >on the behaviour of a new behaviour.
>
> Please, absolutely. I definitely want feedback. I am sure that
> there are many improvements possible that I have not considered.
>
> >>
> >>git fetch git://github.com/jaynel/otp.git gen_stream
> >
> >Thanks. Included in 'pu' with whitespace fixes.
>
> How is this done? I did 'git diff --check' and didn't get
> anything. How do you detect and fix whitespace issues?
There is a hook .git/hooks/pre-commit.sample
that you rename to .git/hooks/pre-commit
and then you can not commit any whitespace errors
(some other problems are checked too).
The argument --check to git diff checks for whitespace errors.
To fix I use git rebase --whitespace=fix
>
> jay
--
/ Raimo Niskanen, Erlang/OTP, Ericsson AB
________________________________________________________________
erlang-patches (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-patches-unsubscribe@erlang.org
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Tue May 18, 2010 12:33 pm |
|
|
|
Guest
|
On Tue, May 18, 2010 at 10:21:40AM +0200, Raimo Niskanen wrote:
> On Mon, May 17, 2010 at 09:29:09AM -0700, Jay Nelson wrote:
> >
> > On May 17, 2010, at 6:33 AM, Raimo Niskanen wrote:
> > >
> > >I do not think so since it does not affect the language
> > >nor the core modules. It may, however, need to be reviewed by
> > >the OTP technical board. We would also like community feedback
> > >on the behaviour of a new behaviour.
> >
> > Please, absolutely. I definitely want feedback. I am sure that
> > there are many improvements possible that I have not considered.
> >
> > >>
> > >>git fetch git://github.com/jaynel/otp.git gen_stream
> > >
> > >Thanks. Included in 'pu' with whitespace fixes.
> >
> > How is this done? I did 'git diff --check' and didn't get
> > anything. How do you detect and fix whitespace issues?
>
> There is a hook .git/hooks/pre-commit.sample
> that you rename to .git/hooks/pre-commit
> and then you can not commit any whitespace errors
> (some other problems are checked too).
>
> The argument --check to git diff checks for whitespace errors.
Oops, you tried that, i realized. Sorry for sloppy reading.
Maybe you did `git diff --check' after commit. Then you
compared the head with the head. Before commit it should
work as intended. After commit you probably want
`git diff --check HEAD^' or against something older
than HEAD^ (the second to latest commit on current branch).
>
> To fix I use git rebase --whitespace=fix
>
> >
> > jay
>
> --
>
> / Raimo Niskanen, Erlang/OTP, Ericsson AB
>
> ________________________________________________________________
> erlang-patches (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:erlang-patches-unsubscribe@erlang.org
--
/ Raimo Niskanen, Erlang/OTP, Ericsson AB
________________________________________________________________
erlang-patches (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-patches-unsubscribe@erlang.org
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Tue May 18, 2010 12:33 pm |
|
|
|
Guest
|
On Tue, May 18, 2010 at 10:21:40AM +0200, Raimo Niskanen wrote:
> On Mon, May 17, 2010 at 09:29:09AM -0700, Jay Nelson wrote:
> >
> > On May 17, 2010, at 6:33 AM, Raimo Niskanen wrote:
> > >
> > >I do not think so since it does not affect the language
> > >nor the core modules. It may, however, need to be reviewed by
> > >the OTP technical board. We would also like community feedback
> > >on the behaviour of a new behaviour.
> >
> > Please, absolutely. I definitely want feedback. I am sure that
> > there are many improvements possible that I have not considered.
> >
> > >>
> > >>git fetch git://github.com/jaynel/otp.git gen_stream
> > >
> > >Thanks. Included in 'pu' with whitespace fixes.
> >
> > How is this done? I did 'git diff --check' and didn't get
> > anything. How do you detect and fix whitespace issues?
>
> There is a hook .git/hooks/pre-commit.sample
> that you rename to .git/hooks/pre-commit
> and then you can not commit any whitespace errors
> (some other problems are checked too).
>
> The argument --check to git diff checks for whitespace errors.
Oops, you tried that, i realized. Sorry for sloppy reading.
Maybe you did `git diff --check' after commit. Then you
compared the head with the head. Before commit it should
work as intended. After commit you probably want
`git diff --check HEAD^' or against something older
than HEAD^ (the second to latest commit on current branch).
>
> To fix I use git rebase --whitespace=fix
>
> >
> > jay
>
> --
>
> / Raimo Niskanen, Erlang/OTP, Ericsson AB
>
> ________________________________________________________________
> erlang-patches (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:erlang-patches-unsubscribe@erlang.org
--
/ Raimo Niskanen, Erlang/OTP, Ericsson AB
________________________________________________________________
erlang-patches (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-patches-unsubscribe@erlang.org
Post received from mailinglist |
|
|
| Back to top |
|
|
|
All times are GMT
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum
|
|
|
Digg It
Del.icio.us
Reddit
Facebook
Stumble Upon
Technorati
