Erlang/OTP Forums

Author Message

<  Erlang questions mailing list  ~  Erlang shows its slow face!

Guest
Posted: Wed Nov 17, 2010 5:47 am Reply with quote
Guest
On 17/11/2010, at 4:34 AM, Edmond Begumisa wrote:

> On Wed, 17 Nov 2010 01:20:51 +1100, Kostis Sagonas <kostis@cs.ntua.gr> wrote:
>
>> There are two basic issues here:
>> (1) The Erlang compiler doesn't really handle list comprehension as
>> native loops; it compiles list comprehensions to recursive
>> functions. lists:seq(1, N) is not special syntax -- unlike Haskell --
>> and almost surely builds an actual list.
>> Yes. This is an issue. Actually, it's more general: in my experience most Erlang programmers do not understand the difference between list comprehensions and lists:foreach/2. Many of them find list comprehension notation so nice that they often use it in places where building the list is not only is not their intention but is completely unnecessary.
>> The compiler is not sufficiently clever to perform deforestation and avoid construction of intermediate lists.
>>
>
> First I'm hearing of this. It really ought to be in one of those green boxes in the documentation.

Let's go around this one more time.

The current Erlang compiler turns

[f(X) || X <- lists:seq(1, N)]

into
<foo>(lists:seq(1, N), ...)
where
<foo>
is a function generated by the compiler.

The compiler *could* make a special case of lists:seq/[2,3]
and generate

<foo>(1, N, ...)

But it doesn't. The actual definition of lists:seq/2 not
only *could* change, in the not too distant past it *has*
changed. The compiler can't inline the definition and
use whether it is now, because it's a remote call (even
if -imported), so it might change at run time.

Haskell compilers and the Mercury compiler *do* perform
deforestation because they are given an unchanging program
to work with and can safely do cross-module inlining.
Erlang can't.

This would be a potentially serious problem if iterating
over integer ranges were something Erlang spent a lot of
time doing.

I just checked an Erlang/OTP release I keep handy for such
things.
~ 1,500,000 raw lines
~ 950,000 SLOW
34 list:seq iterations
18 in two EUNIT tests
3 in other test cases (at least)
13 list:seq not in an ?_assert
~ 3200 Pat <- Exp

The unit tests don't matter for performance. We're left
with about one SLOC in every 72,000 being an iteration of
this kind, or about <- in every 250 being an integer
iteration.

The Erlang/OTP team probably feel they have other maddened
grizzly bears to stun before they bother with something that
seems to turn up mainly in toy benchmarks, and who could blame them?


________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-questions-unsubscribe@erlang.org

Post received from mailinglist
Guest
Posted: Wed Nov 17, 2010 12:49 pm Reply with quote
Guest
Hello Fr=E9d=E9ric,

> Spawning parallel processes is, in my opinion, a false ideal when it =

> comes to algorithms like the ones presented here. In a micro-benchmark=
=

> or in software that is oriented on doing only this task, multiprocessi=
ng =

> might gain you points. In a loaded production system where all the =

> processors are likely already busy, I wouldn't be surprised to see no =
=

> big gain by using multiple processes, and if any, the gain would likel=
y =

> be very hard to predict.
>
> I think it's better to aim for the scaling kind of concurrency, not th=
e =

> speed-up kind. If you use processes for truly concurrent tasks (rather=
=

> than attempting speedups of a single call), You'll benefit from =

> concurrency on a higher level on a server than from each function =

> individually.
>
> This has to be pretty application specific, though.

Hmmm... I see what you mean.

I look at it from an interaction perspective. If the application in =

question is an interactive one, I really hate to see it struggling with =
a =

problem for a long period of time with one of my cores saying "help!" =

while the other one just sits there, and more importantly, while *I* jus=
t =

sit there -- regardless of the algorithm it is working. (Any other =

Photoshop users been annoyed by this?)

For a non-interactive/server application, I can see why you might not wa=
nt =

to do this. But then again, if the muscle is there, and interactive =

clients are waiting... tough call. I guess it's very application specifi=
c =

as you said.

One interesting thing I found from Garcia's results is that for this =

particular problem, crunching it using many processes on a single core w=
as =

basically the same as doing it with one process on a single core. I was =
=

pleasantly surprised by that and didn't expect it.

- Edmond -



On Wed, 17 Nov 2010 02:21:00 +1100, Fr=E9d=E9ric Trottier-H=E9bert =

<fred.hebert@erlang-solutions.com> wrote:

>
> On 2010-11-16, at 09:51 AM, Evans, Matthew wrote:
>
>> Interesting thread...I think we can all take the following from this:=

>>
>> 1) Write your solution in the "Erlang way", using Erlang's strengths =
=

>> such as spawning parallel processes, also HIPE and NIFS where =

>> appropriate.
> Spawning parallel processes is, in my opinion, a false ideal when it =

> comes to algorithms like the ones presented here. In a micro-benchmark=
=

> or in software that is oriented on doing only this task, multiprocessi=
ng =

> might gain you points. In a loaded production system where all the =

> processors are likely already busy, I wouldn't be surprised to see no =
=

> big gain by using multiple processes, and if any, the gain would likel=
y =

> be very hard to predict.
>
> I think it's better to aim for the scaling kind of concurrency, not th=
e =

> speed-up kind. If you use processes for truly concurrent tasks (rather=
=

> than attempting speedups of a single call), You'll benefit from =

> concurrency on a higher level on a server than from each function =

> individually.
>
> This has to be pretty application specific, though.
>
>> 2) Write your solution efficiently - true in all languages, but =

>> especially in Erlang where you could end up doing needless recursion.=

>>
>> 3) Erlang isn't always the best language for all problems. Neither is=
=

>> C, C++, C#, Java or any other language. Choose the language, or a mix=
=

>> of languages, that are the best fit your problem.
>>
>> What I would like to add is that we are trying to get Erlang accepted=
=

>> in a very C/C++ centric development environment. Although time and ti=
me =

>> again Erlang has shown itself a superior language and development =

>> environment for our domain, there is still a big problem in people =

>> accepting it. One of the main stumbling blocks is it is perceived to =
be =

>> "slow". People always say "it won't be faster than a C/C++ solution",=
=

>> or "a language written in a VM will always be slow [rolleyes]".
>>
>> We can argue until we are blue in the face that it will be as good or=
=

>> better when you throw multi-cores in the mix. That there are other =

>> advantages including shorter development time, better debugging, =

>> redundancy, failure detection etc. But overcoming that initial "speed=
" =

>> bias is a tough sell. I'm not saying there is an answer to this, and =
I =

>> know that the development team is doing their best to make Erlang =

>> faster, but we mustn't forget that for many, the "perceived" slowness=
=

>> is one factor that prevents them developing in Erlang.
>>
>
> I think the speed obsession has to be remnants of an education always =
=

> based on time efficiency, especially for programmers coming from the e=
ra =

> where processors were really slow. Whether this explanation is right o=
r =

> not, I don't think it is easily possible to convince someone that 1. h=
is =

> priorities are not the right one and 2. the language he uses isn't the=
=

> best choice for the priorities he has.
>
> We should consider that the burden of proof is ours, not theirs. Any =

> unsubstantiated claim is usually met with skepticism and possibly =

> violence. This is completely normal and natural, in my opinion. The be=
st =

> way to convince someone is with irrefutable proof, which might be why =
=

> the most common suggested way to get a team to accept Erlang is to =

> provide them with an implementation or success stories. And maybe Erla=
ng =

> is no better than what they're using right now, in which case it would=
=

> be a waste of money to train everyone to work in a new environment wit=
h =

> little benefit in the long run.
>
> If people still are not accepting it even after valid proofs, then all=
=

> bets are off: maybe they don't feel like learning new technology, mayb=
e =

> they don't believe that *they* can write efficient Erlang code (and th=
en =

> you have to prove them they can), maybe they think it all comes down t=
o =

> the same anyway (project success isn't language dependent? which could=
=

> be true in many cases), etc. I'm not sure much can be done except =

> leading by example, in any case.
>
>> /Rant over
>>
>> Matt
>>
>
>
> --
> Fred H=E9bert
> http://www.erlang-solutions.com
>
>
>
>


-- =

Using Opera's revolutionary e-mail client: http://www.opera.com/mail/

________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-questions-unsubscribe@erlang.org

Post received from mailinglist
Guest
Posted: Wed Nov 17, 2010 1:23 pm Reply with quote
Guest
> But overcoming that initial "speed" bias is a tough sell. I'm not saying
> there is an answer to this, and I know that the development team is
> doing their best to make Erlang faster, but we mustn't forget that for
> many, the "perceived" slowness is one factor that prevents them
> developing in Erlang.

I'm not sure that speed matters that much to people anymore (at least not
for application developers, it obviously matters for system developers).
Remember when Java and (classic) Visual Basic came out? There were many
concerns about speed, but in a short period of time these languages
dominated in application development (especially for business
applications) because people had other higher priorities like
ease-of-learning and time-to-market* as Richard pointed out.

Personally, I think it's two other factors that are mainly causing
Erlang's slower than expected penetration in general commercial
application development (as opposed to it's well known niches)...

1. Erlang is perceived as "weird"

A lot of people I've forwarded links to on Erlang just find it too
strange! I'm not taking just syntax, concepts like variables that don't
change are just too odd for them. And things they find familiar are done
too differently. "How do I create an object-model?" they ask.

2. Not "main-stream"

Realising that is a chicken-and-egg issue, one problem in application
development is people feel comfort in numbers. If something isn't used by
a good swath of software companies people are very hard to convince to use
it. If it is wide-spread people graduate it to the mythical status of a
"industry standard" and insist on using it even when there are far better
tools for the problem space. JavaScript is a good example. I'm a huge fan
of JavaScript for client-side development but I'm seeing it popup in the
most unusual places, places which are inappropriate in my view.

* Those two particular issues have a big impact on cost of development.
Shorter development time obviously impacts the investment amount.
Ease-of-learning means you can higher less-experienced, less-skilled
programmers and more importantly, pay lower salaries. It wouldn't surprise
me if the average cost to higher a competent Erlang programmer for 6
months is double that for a competent C# programmer! Sad, but these things
matter. IMO, commercial application developers pay dearly in the end for
choosing the wrong tools for the wrong reasons.

- Edmond -


--
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/

________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-questions-unsubscribe@erlang.org

Post received from mailinglist
Guest
Posted: Wed Nov 17, 2010 3:02 pm Reply with quote
Guest
Aaaahhh,

> The compiler *could* make a special case of lists:seq/[2,3]
> and generate
>
> <foo>(1, N, ...)
>
> But it doesn't.

OK. I erroneously thought that this *was* the case.

> The compiler can't inline the definition and
> use whether it is now, because it's a remote call (even
> if -imported), so it might change at run time.

I see it now Smile

- Edmond -


On Wed, 17 Nov 2010 16:45:09 +1100, Richard O'Keefe <ok@cs.otago.ac.nz>
wrote:

>
> On 17/11/2010, at 4:34 AM, Edmond Begumisa wrote:
>
>> On Wed, 17 Nov 2010 01:20:51 +1100, Kostis Sagonas <kostis@cs.ntua.gr>
>> wrote:
>>
>>> There are two basic issues here:
>>> (1) The Erlang compiler doesn't really handle list comprehension as
>>> native loops; it compiles list comprehensions to recursive
>>> functions. lists:seq(1, N) is not special syntax -- unlike Haskell
>>> --
>>> and almost surely builds an actual list.
>>> Yes. This is an issue. Actually, it's more general: in my experience
>>> most Erlang programmers do not understand the difference between list
>>> comprehensions and lists:foreach/2. Many of them find list
>>> comprehension notation so nice that they often use it in places where
>>> building the list is not only is not their intention but is completely
>>> unnecessary.
>>> The compiler is not sufficiently clever to perform deforestation and
>>> avoid construction of intermediate lists.
>>>
>>
>> First I'm hearing of this. It really ought to be in one of those green
>> boxes in the documentation.
>
> Let's go around this one more time.
>
> The current Erlang compiler turns
>
> [f(X) || X <- lists:seq(1, N)]
>
> into
> <foo>(lists:seq(1, N), ...)
> where
> <foo>
> is a function generated by the compiler.
>
> The compiler *could* make a special case of lists:seq/[2,3]
> and generate
>
> <foo>(1, N, ...)
>
> But it doesn't. The actual definition of lists:seq/2 not
> only *could* change, in the not too distant past it *has*
> changed. The compiler can't inline the definition and
> use whether it is now, because it's a remote call (even
> if -imported), so it might change at run time.
>
> Haskell compilers and the Mercury compiler *do* perform
> deforestation because they are given an unchanging program
> to work with and can safely do cross-module inlining.
> Erlang can't.
>
> This would be a potentially serious problem if iterating
> over integer ranges were something Erlang spent a lot of
> time doing.
>
> I just checked an Erlang/OTP release I keep handy for such
> things.
> ~ 1,500,000 raw lines
> ~ 950,000 SLOW
> 34 list:seq iterations
> 18 in two EUNIT tests
> 3 in other test cases (at least)
> 13 list:seq not in an ?_assert
> ~ 3200 Pat <- Exp
>
> The unit tests don't matter for performance. We're left
> with about one SLOC in every 72,000 being an iteration of
> this kind, or about <- in every 250 being an integer
> iteration.
>
> The Erlang/OTP team probably feel they have other maddened
> grizzly bears to stun before they bother with something that
> seems to turn up mainly in toy benchmarks, and who could blame them?
>
>
> ________________________________________________________________
> erlang-questions (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:erlang-questions-unsubscribe@erlang.org
>


--
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/

________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-questions-unsubscribe@erlang.org

Post received from mailinglist
Guest
Posted: Wed Nov 17, 2010 3:19 pm Reply with quote
Guest
That was supposed to be 'hire' not 'higher' in that last paragraph. My
auto-spell-corrector goes insane at times Smile

- Edmond -


On Thu, 18 Nov 2010 00:21:21 +1100, Edmond Begumisa
<ebegumisa@hysteria-tech.com> wrote:

>> But overcoming that initial "speed" bias is a tough sell. I'm not
>> saying there is an answer to this, and I know that the development team
>> is doing their best to make Erlang faster, but we mustn't forget that
>> for many, the "perceived" slowness is one factor that prevents them
>> developing in Erlang.
>
> I'm not sure that speed matters that much to people anymore (at least
> not for application developers, it obviously matters for system
> developers). Remember when Java and (classic) Visual Basic came out?
> There were many concerns about speed, but in a short period of time
> these languages dominated in application development (especially for
> business applications) because people had other higher priorities like
> ease-of-learning and time-to-market* as Richard pointed out.
>
> Personally, I think it's two other factors that are mainly causing
> Erlang's slower than expected penetration in general commercial
> application development (as opposed to it's well known niches)...
>
> 1. Erlang is perceived as "weird"
>
> A lot of people I've forwarded links to on Erlang just find it too
> strange! I'm not taking just syntax, concepts like variables that don't
> change are just too odd for them. And things they find familiar are done
> too differently. "How do I create an object-model?" they ask.
>
> 2. Not "main-stream"
>
> Realising that is a chicken-and-egg issue, one problem in application
> development is people feel comfort in numbers. If something isn't used
> by a good swath of software companies people are very hard to convince
> to use it. If it is wide-spread people graduate it to the mythical
> status of a "industry standard" and insist on using it even when there
> are far better tools for the problem space. JavaScript is a good
> example. I'm a huge fan of JavaScript for client-side development but
> I'm seeing it popup in the most unusual places, places which are
> inappropriate in my view.
>
> * Those two particular issues have a big impact on cost of development.
> Shorter development time obviously impacts the investment amount.
> Ease-of-learning means you can higher less-experienced, less-skilled
> programmers and more importantly, pay lower salaries. It wouldn't
> surprise me if the average cost to higher a competent Erlang programmer
> for 6 months is double that for a competent C# programmer! Sad, but
> these things matter. IMO, commercial application developers pay dearly
> in the end for choosing the wrong tools for the wrong reasons.
>
> - Edmond -
>
>


--
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/

________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-questions-unsubscribe@erlang.org

Post received from mailinglist
wuji
Posted: Mon Sep 03, 2012 7:28 am Reply with quote
User Joined: 10 Aug 2012 Posts: 654
Membership Chair."Instead, Guy said she planned to attend Cornell's program program [h2]imitation designer *beep*[/h2] program in Washington, D.C., in the fall, where she would
in both classes and have an internship, in order to to replica designer *beep* to be close to her home state to fulfill her
Wheatley, director of media relations for Cornell University, said, "That "That [h1]replica designer *beep*[/h1] "That letter was never meant to be public. It was
private letter between her and her sisters. Someone misbehaved in in cheap real jordans in releasing it."Wheatley said she is "staggered" by the involvement
Cornell students, and said their ability to juggle is something something buy real jordans something that they have been doing since they were in
school and continued successfully into college.But it's uncommon for many many [h4]replica designer *beep*[/h4] many state representatives to attend school while fulfilling their duties,
View user's profile Send private message

Display posts from previous:  

All times are GMT
Page 4 of 4
Goto page Previous  1, 2, 3, 4
This forum is locked: you cannot post, reply to, or edit topics.

Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum