| Author |
Message |
|
| Guest |
 Posted: Wed Jul 28, 2010 1:14 pm |
 |
|
|
Guest
|
On Sun, Jul 4, 2010 at 3:19 PM, Paul Guyot <pguyot@kallisys.net> wrote:
> Hello,
>
> beam_dict:string and the whole string table code is based on naive string search and compiling a module with thousands of binary strings takes several minutes. The following patch replaces the code with calls to binary:match/2.
>
> git fetch git://github.com/pguyot/otp.git pg/optimize_beam_dict_string_table
>
> http://github.com/pguyot/otp/commit/13ca6aa7b74c7887ae6c3a42b35769524af4903a
>
Can you give some more information about the claims in the commit
message?
I did the following in $ERL_TOP/release/tests/stdlib_test
(using the R14A compiler):
erlc +time -I ../test_server re_testoutput1_split_test.erl
The interesting part of the output are the following lines:
core_module : 10.26 s 60987.5 kB
v3_codegen : 5.83 s 17908.4 kB
beam_asm : 0.46 s 2.5 kB
The module contains several thousand strings
that are used in bit syntax matching. The core_module
and v3_codegen passes are very slow. The beam_asm
pass is slower than usual, but fast compared to the
slow passes.
Do you have a module that will make the beam_asm
pass very slow?
If not, the change is still interesting because the new
code is shorter and easier to understand, but the
commit message must be revised.
Regarding the implementation, it will probably be
faster to append new strings to the string table
like this:
NewDict = Dict#asm{strings = <<Strings/binary,StrBin/binary>>,
string_offset=NextOffset+byte_size(StrBin)},
Regarding the test case, a better place for it should be the
compilation_SUITE module.
(The name of the test modules may not be the best, but
compile_SUITE (mainly) tests different compiler options,
while compilation_SUITE tests miscellanous language features.)
--
Björn Gustavsson, Erlang/OTP, Ericsson AB
________________________________________________________________
erlang-patches (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-patches-unsubscribe@erlang.org
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Wed Jul 28, 2010 1:15 pm |
|
|
|
Guest
|
On Sun, Jul 4, 2010 at 3:19 PM, Paul Guyot <pguyot@kallisys.net> wrote:
> Hello,
>
> beam_dict:string and the whole string table code is based on naive string search and compiling a module with thousands of binary strings takes several minutes. The following patch replaces the code with calls to binary:match/2.
>
> git fetch git://github.com/pguyot/otp.git pg/optimize_beam_dict_string_table
>
> http://github.com/pguyot/otp/commit/13ca6aa7b74c7887ae6c3a42b35769524af4903a
>
Can you give some more information about the claims in the commit
message?
I did the following in $ERL_TOP/release/tests/stdlib_test
(using the R14A compiler):
erlc +time -I ../test_server re_testoutput1_split_test.erl
The interesting part of the output are the following lines:
core_module : 10.26 s 60987.5 kB
v3_codegen : 5.83 s 17908.4 kB
beam_asm : 0.46 s 2.5 kB
The module contains several thousand strings
that are used in bit syntax matching. The core_module
and v3_codegen passes are very slow. The beam_asm
pass is slower than usual, but fast compared to the
slow passes.
Do you have a module that will make the beam_asm
pass very slow?
If not, the change is still interesting because the new
code is shorter and easier to understand, but the
commit message must be revised.
Regarding the implementation, it will probably be
faster to append new strings to the string table
like this:
NewDict = Dict#asm{strings = <<Strings/binary,StrBin/binary>>,
string_offset=NextOffset+byte_size(StrBin)},
Regarding the test case, a better place for it should be the
compilation_SUITE module.
(The name of the test modules may not be the best, but
compile_SUITE (mainly) tests different compiler options,
while compilation_SUITE tests miscellanous language features.)
--
Björn Gustavsson, Erlang/OTP, Ericsson AB
________________________________________________________________
erlang-patches (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-patches-unsubscribe@erlang.org
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Wed Jul 28, 2010 3:26 pm |
|
|
|
Guest
|
|
| Back to top |
|
| Guest |
| Posted: Wed Jul 28, 2010 3:27 pm |
|
|
|
Guest
|
|
| Back to top |
|
| Guest |
| Posted: Thu Jul 29, 2010 7:36 am |
|
|
|
Guest
|
2010/7/28 Paul Guyot <pguyot@kallisys.net>:
> With 1000 of such clauses:
> lists:foreach(fun(_) -> <<I:80>> = crypto:rand_bytes(10), S = erlang:integer_to_list(I, 16), io:format("f(<<\"~s\">>) -> atom~s;\n", [S, S]) end, lists:seq(0, 1000)).
>
> I get:
> core_module : 0.05 s 25922.5 kB
> v3_codegen : 0.09 s 6660.6 kB
> beam_asm : 0.28 s 2.6 kB
>
> vs
> core_module : 0.06 s 20558.1 kB
> v3_codegen : 0.08 s 6617.9 kB
> beam_asm : 0.03 s 1.7 kB
>
I get similar figures (without any native code).
>> Regarding the implementation, it will probably be
>> faster to append new strings to the string table
>> like this:
>>
>> NewDict = Dict#asm{strings = <<Strings/binary,StrBin/binary>>,
>> string_offset=NextOffset+byte_size(StrBin)},
Compiling a module with 5000 clauses I can confirm that this
version is faster.
>>
>> Regarding the test case, a better place for it should be the
>> compilation_SUITE module.
>>
>> (The name of the test modules may not be the best, but
>> compile_SUITE (mainly) tests different compiler options,
>> while compilation_SUITE tests miscellanous language features.)
>
> Do you want me to update the branch ?
Normally I would ask you to update the branch yourself.
But since I have already done measurements using your
branch, it was easy enough for me to do the changes and
revise the commit message to emphasize the simplification
of the code rather than the optimization.
Here is the updated version (so far only in my own
github repository):
http://github.com/bjorng/otp/commit/173d1fd1c3fef385f73accc4b2bbb1b6f92ac3f5
If you approve this version, I will include it in 'pu' later
today.
--
Björn Gustavsson, Erlang/OTP, Ericsson AB
________________________________________________________________
erlang-patches (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-patches-unsubscribe@erlang.org
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Thu Jul 29, 2010 7:36 am |
|
|
|
Guest
|
2010/7/28 Paul Guyot <pguyot@kallisys.net>:
> With 1000 of such clauses:
> lists:foreach(fun(_) -> <<I:80>> = crypto:rand_bytes(10), S = erlang:integer_to_list(I, 16), io:format("f(<<\"~s\">>) -> atom~s;\n", [S, S]) end, lists:seq(0, 1000)).
>
> I get:
> core_module : 0.05 s 25922.5 kB
> v3_codegen : 0.09 s 6660.6 kB
> beam_asm : 0.28 s 2.6 kB
>
> vs
> core_module : 0.06 s 20558.1 kB
> v3_codegen : 0.08 s 6617.9 kB
> beam_asm : 0.03 s 1.7 kB
>
I get similar figures (without any native code).
>> Regarding the implementation, it will probably be
>> faster to append new strings to the string table
>> like this:
>>
>> NewDict = Dict#asm{strings = <<Strings/binary,StrBin/binary>>,
>> string_offset=NextOffset+byte_size(StrBin)},
Compiling a module with 5000 clauses I can confirm that this
version is faster.
>>
>> Regarding the test case, a better place for it should be the
>> compilation_SUITE module.
>>
>> (The name of the test modules may not be the best, but
>> compile_SUITE (mainly) tests different compiler options,
>> while compilation_SUITE tests miscellanous language features.)
>
> Do you want me to update the branch ?
Normally I would ask you to update the branch yourself.
But since I have already done measurements using your
branch, it was easy enough for me to do the changes and
revise the commit message to emphasize the simplification
of the code rather than the optimization.
Here is the updated version (so far only in my own
github repository):
http://github.com/bjorng/otp/commit/173d1fd1c3fef385f73accc4b2bbb1b6f92ac3f5
If you approve this version, I will include it in 'pu' later
today.
--
Björn Gustavsson, Erlang/OTP, Ericsson AB
________________________________________________________________
erlang-patches (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-patches-unsubscribe@erlang.org
Post received from mailinglist |
|
|
| Back to top |
|
| wailian |
| Posted: Tue Mar 20, 2012 2:40 am |
|
|
|
Guest
|
| The shoes absolutely are an avant-garde day time day admiration and do amazing problems even although access just about any apparel! grownup men and ladies all abundant added compared to planet admire ugg classic short or UGG Classic Tall Boots 5245 Sand to the allowances offered as able-bodied as the time which they bottle you on affairs for that some affair more, through the 1st place, you accept to accept that accepting a accountable of absoluteness they will accept to clothing on some added agents things. |
|
|
| Back to top |
|
|
|
All times are GMT
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum
|
|
|
Digg It
Del.icio.us
Reddit
Facebook
Stumble Upon
Technorati
