| Author |
Message |
< Erlang questions mailing list ~ [ANN] Erlson - dynamic name-value dictionary data type and s |
| Guest |
 Posted: Mon Aug 29, 2011 4:50 am |
 |
|
|
Guest
|
Hi list,
It is my pleasure to announce Erlson - a proof of concept
implementation of dynamic dictionary data type. It can be best
described by examples:
X = #{}, % create an empty dictionary
D = #{foo = 1, bar = "abc", fum}, % associate fields 'foo' with
1, 'bar' with "abc" and 'fum' with 'true'
1 = D.foo, % access dictionary element
D1 = D#{baz = #{fum = #{i = 0}}}, % add nested dictionaries to
dictionary D
0 = D1.baz.fum.i, % access elements of the nested dictionary
D2 = D1#{baz.fum.i = 100, baz.fum.j = "new nested value"}. %
modify elements of the nested dictionary
...
erlson:to_json(D2). % convert Erlson dictionary to JSON iolist()
D = erlson:from_json(Json). % create Erlson dictionary from JSON iolist()
...
D = erlson:from_proplist(L). % create Erlson dictionary from a proplist
D = erlson:from_nested_proplist(L). % create nested Erlson
dictionary from a nested proplist
Erlson implementation is fully working and includes syntax support,
runtime library and rebar packaging. The source code is available on
GitHub: https://github.com/alavrik/erlson
Internally, Erlson dictionaries are represented as orddicts, i.e.
ordered lists of {atom(), any()} pairs. This way, they can be easily
printed, manipulated and explored using conventional methods. For
example, using the standard "orddict" and "proplists" libraries. It is
also possible to define types for Erlson dictionaries and have them
checked by Dialyzer.
Erlson dictionary syntax can be used in both .erl modules and Erlang
shell. Its implementation is based on a customized version of
"erl_parse.yrl", which extends several grammar rules and overloads
existing syntax elements used by Erlang records. Because there was no
need to introduce new syntax elements, the implementation turned out
to be quite simple.
There is a catch, though. The current Erlson implementation is based
on the original "erl_parse.yrl" from R14B03 release. This makes Erlson
incompatible with other Erlang releases if they have a different
version of "erl_parse.yrl".
Although it would be easy to provide Erlson-enabled "erl_parse.yrl"
version for every Erlang release, I was thinking about of a less hacky
and more reliable solution. And this brings me to the question:
Would it be useful if Erlson or some subset/superset of it became a
part of the Erlang language? What do you think?
Anton
_______________________________________________
erlang-questions mailing list
erlang-questions@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Mon Aug 29, 2011 5:05 am |
|
|
|
Guest
|
Looks great. Glancing at the code, erlson:from_proplist if your proplist key is not an atom will not work, or will it? It would be nice if it automatically converted from lists and binaries.
Sergej
On Mon, Aug 29, 2011 at 6:50 AM, Anton Lavrik <alavrik@piqi.org (alavrik@piqi.org)> wrote:
Quote: Hi list,
It is my pleasure to announce Erlson - a proof of concept
implementation of dynamic dictionary data type. It can be best
described by examples:
|
|
|
| Back to top |
|
| Guest |
| Posted: Mon Aug 29, 2011 5:35 am |
|
|
|
Guest
|
On Mon, Aug 29, 2011 at 12:05 AM, Rapsey <rapsey@gmail.com> wrote:
> Looks great. Glancing at the code, erlson:from_proplist if your proplist key
> is not an atom will not work, or will it? It would be nice if it
> automatically converted from lists and binaries.
Yes, erlson:from_proplist() accepts only atoms as keys. Otherwise, it
would be inconsistent with property lists as defined by the
"proplists" module. Here's a quote from the documentation:
"Property lists are ordinary lists containing entries in the form of
either tuples, whose first elements are keys used for lookup and
insertion, or atoms, which work as shorthand for tuples {Atom, true}
."
Anton
_______________________________________________
erlang-questions mailing list
erlang-questions@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Mon Aug 29, 2011 6:02 am |
|
|
|
Guest
|
On 08/29/11 08:35, Anton Lavrik wrote:
> On Mon, Aug 29, 2011 at 12:05 AM, Rapsey<rapsey@gmail.com> wrote:
>> Looks great. Glancing at the code, erlson:from_proplist if your proplist key
>> is not an atom will not work, or will it? It would be nice if it
>> automatically converted from lists and binaries.
>
> Yes, erlson:from_proplist() accepts only atoms as keys. Otherwise, it
> would be inconsistent with property lists as defined by the
> "proplists" module. Here's a quote from the documentation:
>
> "Property lists are ordinary lists containing entries in the form of
> either tuples, whose first elements are keys used for lookup and
> insertion, or atoms, which work as shorthand for tuples {Atom, true}."
I really do not see how the above quote implies that keys are atoms in
proplists. My reading is that if tuples are used as properties, their
keys can actually be any term. You may be right, but not because of
this quote.
Kostis
_______________________________________________
erlang-questions mailing list
erlang-questions@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Mon Aug 29, 2011 6:08 am |
|
|
|
Guest
|
On Mon, Aug 29, 2011 at 1:02 AM, Kostis Sagonas <kostis@cs.ntua.gr> wrote:
> On 08/29/11 08:35, Anton Lavrik wrote:
>>
>> On Mon, Aug 29, 2011 at 12:05 AM, Rapsey<rapsey@gmail.com> |
|
|
| Back to top |
|
| Guest |
| Posted: Mon Aug 29, 2011 6:12 am |
|
|
|
Guest
|
On 29/08/2011, at 6:02 PM, Kostis Sagonas wrote:
> I really do not see how the above quote implies that keys are atoms in proplists. My reading is that if tuples are used as properties, their keys can actually be any term. You may be right, but not because of this quote.
I read the proplist documentation the same way Kostis Sagonas does.
Perhaps more importantly, we are told over and over again that
Key = term()
not that Key = atom().
And look at the code for lookup, edited for readability:
lookup(Key, [P | Ps]) ->
if is_atom(P), P =:= Key ->
{Key, true}
; tuple_size(P) >= 1, element(1, P) =:= Key ->
%% Note that <code>Key</code> does not have to be an atom in this case.
P
; true ->
lookup(Key, Ps)
end;
lookup(_Key, []) ->
none.
The comment there could hardly be more explicit.
Of course, as explained at length in the proposals and on this mailing list,
the ultra-lightweight "frames"/"proper structs" that Erlang needs ought to
restrict *their* keys to being atoms.
_______________________________________________
erlang-questions mailing list
erlang-questions@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Mon Aug 29, 2011 6:33 am |
|
|
|
Guest
|
|
| Back to top |
|
| uwiger |
| Posted: Mon Aug 29, 2011 7:17 am |
|
|
|
User
Joined: 03 Jul 2006
Posts: 604
Location: Sweden
|
On 29 Aug 2011, at 06:50, Anton Lavrik wrote:
> Hi list,
>
> It is my pleasure to announce Erlson - a proof of concept
> implementation of dynamic dictionary data type.
This library was one of the entries in the Spawnfest competition, and received favourable mention from several of the judges (including me). I like the syntax - it feels like a good, minimal extension to the existing language.
Personally, I would really like to see a special data type for this, as well as proper support for pattern matching and guards. I'm wary of yet another syntactic-sugar solution, overloading existing data types.
This is in no way meant as criticism of the work so far - rather as a suggestion to grab the opportunity and implement full support for dynamic dictionaries (hashes, or whatever we want to call them).
As always, there may be real, substantial objections that I have overlooked. If so, the good members of this list will surely present them. 
BR,
Ulf W
Ulf Wiger, CTO, Erlang Solutions, Ltd.
http://erlang-solutions.com
_______________________________________________
erlang-questions mailing list
erlang-questions@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions
Post received from mailinglist |
|
|
| Back to top |
  
|
| Guest |
| Posted: Mon Aug 29, 2011 11:26 pm |
|
|
|
Guest
|
May I suggest switching the decoded JSON format to that used by ejson (https://github.com/benoitc/ejson) and jiffy (https://github.com/davisp/jiffy) instead of the format used by mochiweb/mochijson2. Most of the projects that make heavy use of JSON seem to be moving to the format I'm suggesting.
Juanjo
On Mon, Aug 29, 2011 at 1:50 AM, Anton Lavrik <alavrik@piqi.org (alavrik@piqi.org)> wrote:
Quote: Hi list,
It is my pleasure to announce Erlson - a proof of concept
implementation of dynamic dictionary data type. It can be best
described by examples:
X = #{}, % create an empty dictionary
D = #{foo = 1, bar = "abc", fum}, % associate fields 'foo' with
1, 'bar' with "abc" and 'fum' with 'true'
1 = D.foo, % access dictionary element
D1 = D#{baz = #{fum = #{i = 0}}}, % add nested dictionaries to
dictionary D
0 = D1.baz.fum.i, % access elements of the nested dictionary
D2 = D1#{baz.fum.i = 100, baz.fum.j = "new nested value"}. %
modify elements of the nested dictionary
...
erlson:to_json(D2). % convert Erlson dictionary to JSON iolist()
D = erlson:from_json(Json). % create Erlson dictionary from JSON iolist()
...
D = erlson:from_proplist(L). % create Erlson dictionary from a proplist
D = erlson:from_nested_proplist(L). % create nested Erlson
dictionary from a nested proplist
Erlson implementation is fully working and includes syntax support,
runtime library and rebar packaging. The source code is available on
GitHub: https://github.com/alavrik/erlson
Internally, Erlson dictionaries are represented as orddicts, i.e.
ordered lists of {atom(), any()} pairs. This way, they can be easily
printed, manipulated and explored using conventional methods. For
example, using the standard "orddict" and "proplists" libraries. It is
also possible to define types for Erlson dictionaries and have them
checked by Dialyzer.
Erlson dictionary syntax can be used in both .erl modules and Erlang
shell. Its implementation is based on a customized version of
"erl_parse.yrl", which extends several grammar rules and overloads
existing syntax elements used by Erlang records. Because there was no
need to introduce new syntax elements, the implementation turned out
to be quite simple.
There is a catch, though. The current Erlson implementation is based
on the original "erl_parse.yrl" from R14B03 release. This makes Erlson
incompatible with other Erlang releases if they have a different
version of "erl_parse.yrl".
Although it would be easy to provide Erlson-enabled "erl_parse.yrl"
version for every Erlang release, I was thinking about of a less hacky
and more reliable solution. And this brings me to the question:
Would it be useful if Erlson or some subset/superset of it became a
part of the Erlang language? What do you think?
Anton
_______________________________________________
erlang-questions mailing list
erlang-questions@erlang.org (erlang-questions@erlang.org)
http://erlang.org/mailman/listinfo/erlang-questions
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Mon Aug 29, 2011 11:36 pm |
|
|
|
Guest
|
> May I suggest switching the decoded JSON format to that used by ejson
> (https://github.com/benoitc/ejson) and jiffy
> (https://github.com/davisp/jiffy) instead of the format used by
> mochiweb/mochijson2. Most of the projects that make heavy use of JSON seem
> to be moving to the format I'm suggesting.
A tuple of an array of tuples seems less than elegant Why not just a
tuple of tuples? Is scanning through a tuple significantly slower than
scanning an array?
As far as formats go, this is a new one to me. Most projects I
interact with seem to use either mochijson or mochijson2. I'd love for
there to be a canonical one though.
jack.
_______________________________________________
erlang-questions mailing list
erlang-questions@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Mon Aug 29, 2011 11:55 pm |
|
|
|
Guest
|
On 08/30/2011 01:36 AM, Jack Moffitt wrote:
>> May I suggest switching the decoded JSON format to that used by ejson
>> (https://github.com/benoitc/ejson) and jiffy
>> (https://github.com/davisp/jiffy) instead of the format used by
>> mochiweb/mochijson2. Most of the projects that make heavy use of JSON seem
>> to be moving to the format I'm suggesting.
>
> A tuple of an array of tuples seems less than elegant Why not just a
> tuple of tuples? Is scanning through a tuple significantly slower than
> scanning an array?
This should explain the whys:
http://www.erlang.org/eeps/eep-0018.html
> As far as formats go, this is a new one to me. Most projects I
> interact with seem to use either mochijson or mochijson2. I'd love for
> there to be a canonical one though.
All recent JSON projects follow that EEP as far as I know. If OTP gets
JSON support then your code will be working with the OTP implementation.
It's also less annoying to write than {struct, ...} because it's trying
(and not failing on me yet) to be a 1:1 match with Erlang types or when
not possible to give expected results.
Personally I prefer jsx (https://github.com/talentdeficit/jsx) simply
because it's not a NIF and thus doesn't impact the scheduler. But your
needs and requirements may vary.
--
Lo |
|
|
| Back to top |
|
| Guest |
| Posted: Tue Aug 30, 2011 12:12 am |
|
|
|
Guest
|
On Mon, Aug 29, 2011 at 4:53 PM, Loïc Hoguin <essen@dev-extend.eu> wrote:
> On 08/30/2011 01:36 AM, Jack Moffitt wrote:
>>> May I suggest switching the decoded JSON format to that used by ejson
>>> (https://github.com/benoitc/ejson) and jiffy
>>> (https://github.com/davisp/jiffy) instead of the format used by
>>> mochiweb/mochijson2. Most of the projects that make heavy use of JSON seem
>>> to be moving to the format I'm suggesting.
>>
>> A tuple of an array of tuples seems less than elegant Why not just a
>> tuple of tuples? Is scanning through a tuple significantly slower than
>> scanning an array?
>
> This should explain the whys:
>
> http://www.erlang.org/eeps/eep-0018.html
>
>> As far as formats go, this is a new one to me. Most projects I
>> interact with seem to use either mochijson or mochijson2. I'd love for
>> there to be a canonical one though.
>
> All recent JSON projects follow that EEP as far as I know. If OTP gets
> JSON support then your code will be working with the OTP implementation.
> It's also less annoying to write than {struct, ...} because it's trying
> (and not failing on me yet) to be a 1:1 match with Erlang types or when
> not possible to give expected results.
>
> Personally I prefer jsx (https://github.com/talentdeficit/jsx) simply
> because it's not a NIF and thus doesn't impact the scheduler. But your
> needs and requirements may vary.
It would be basically trivial to add {proplist()} support to
mochijson2 as an addition to {struct, proplist()}. There's not much of
a difference really, it's still wrapping a tuple around it but just
without a tag. I'll have a look at that.
-bob
_______________________________________________
erlang-questions mailing list
erlang-questions@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Tue Aug 30, 2011 1:09 am |
|
|
|
Guest
|
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 08/29/2011 05:12 PM, Bob Ippolito wrote:
> On Mon, Aug 29, 2011 at 4:53 PM, Loïc Hoguin <essen@dev-extend.eu> wrote:
>> On 08/30/2011 01:36 AM, Jack Moffitt wrote:
>>>> May I suggest switching the decoded JSON format to that used by ejson
>>>> (https://github.com/benoitc/ejson) and jiffy
>>>> (https://github.com/davisp/jiffy) instead of the format used by
>>>> mochiweb/mochijson2. Most of the projects that make heavy use of JSON seem
>>>> to be moving to the format I'm suggesting.
>>>
>>> A tuple of an array of tuples seems less than elegant Why not just a
>>> tuple of tuples? Is scanning through a tuple significantly slower than
>>> scanning an array?
>>
>> This should explain the whys:
>>
>> http://www.erlang.org/eeps/eep-0018.html
>>
>>> As far as formats go, this is a new one to me. Most projects I
>>> interact with seem to use either mochijson or mochijson2. I'd love for
>>> there to be a canonical one though.
>>
>> All recent JSON projects follow that EEP as far as I know. If OTP gets
>> JSON support then your code will be working with the OTP implementation.
>> It's also less annoying to write than {struct, ...} because it's trying
>> (and not failing on me yet) to be a 1:1 match with Erlang types or when
>> not possible to give expected results.
>>
>> Personally I prefer jsx (https://github.com/talentdeficit/jsx) simply
>> because it's not a NIF and thus doesn't impact the scheduler. But your
>> needs and requirements may vary.
>
> It would be basically trivial to add {proplist()} support to
> mochijson2 as an addition to {struct, proplist()}. There's not much of
> a difference really, it's still wrapping a tuple around it but just
> without a tag. I'll have a look at that.
>
> -bob
> _______________________________________________
> erlang-questions mailing list
> erlang-questions@erlang.org
> http://erlang.org/mailman/listinfo/erlang-questions
An old version of Couchbeam accomplished that conversion thusly, if that
helps:
json_encode(V) ->
Handler =
fun({L}) when is_list(L) ->
{struct,L};
(Bad) ->
exit({json_encode, {bad_term, Bad}})
end,
(mochijson2:encoder([{handler, Handler}]))(V).
json_decode(V) ->
try (mochijson2:decoder([{object_hook, fun({struct,L}) -> {L} end}]))(V)
catch
_Type:_Error ->
throw({invalid_json,V})
end.
- --
James Aimonetti
Distributed Systems Engineer / DJ MC_
2600hz | http://2600hz.com
sip:james@2600hz.com
tel: 415.886.7905
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iQEcBAEBAgAGBQJOXDdOAAoJENc77s1OYoGg4D8H/2j63QYpyFRymqz62rhmlquf
5jCf9NoMpzYfSsJuC9fJhcvOtbpdr/Cx+62Y3GRDi0UMntdzvS0IwNmJvqH9uCer
WQ7gpPY+0tGxGfxDlMTnoBvgcfQM6bbjjMPFqN+dgDinVliAgwRhZpaJgd0hgvOm
g0Xb6ngK+pvueJV0cfTBsTFmMgI89rxJMb4yUFc0V54XctQTsAZhaxmP429jt0mP
e0YGQ0ZvAQaHM+CjRYuUlKHUrCUNmkXmgmjk54L9S0Ic5PZLpJMdpMWq6FWsMrIt
D7klF7x8E2v3w0gCHsPRS7n/XPV0Sp0sEuMz75Kifs5nsAuNUmLX5vG77sXuZ5s=
=QFs/
-----END PGP SIGNATURE-----
_______________________________________________
erlang-questions mailing list
erlang-questions@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Tue Aug 30, 2011 1:19 am |
|
|
|
Guest
|
On Mon, Aug 29, 2011 at 6:05 PM, James Aimonetti <james@2600hz.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
> On 08/29/2011 05:12 PM, Bob Ippolito wrote:
>> On Mon, Aug 29, 2011 at 4:53 PM, Loïc Hoguin <essen@dev-extend.eu> wrote:
>>> On 08/30/2011 01:36 AM, Jack Moffitt wrote:
>>>>> May I suggest switching the decoded JSON format to that used by ejson
>>>>> (https://github.com/benoitc/ejson) and jiffy
>>>>> (https://github.com/davisp/jiffy) instead of the format used by
>>>>> mochiweb/mochijson2. Most of the projects that make heavy use of JSON seem
>>>>> to be moving to the format I'm suggesting.
>>>>
>>>> A tuple of an array of tuples seems less than elegant Why not just a
>>>> tuple of tuples? Is scanning through a tuple significantly slower than
>>>> scanning an array?
>>>
>>> This should explain the whys:
>>>
>>> http://www.erlang.org/eeps/eep-0018.html
>>>
>>>> As far as formats go, this is a new one to me. Most projects I
>>>> interact with seem to use either mochijson or mochijson2. I'd love for
>>>> there to be a canonical one though.
>>>
>>> All recent JSON projects follow that EEP as far as I know. If OTP gets
>>> JSON support then your code will be working with the OTP implementation.
>>> It's also less annoying to write than {struct, ...} because it's trying
>>> (and not failing on me yet) to be a 1:1 match with Erlang types or when
>>> not possible to give expected results.
>>>
>>> Personally I prefer jsx (https://github.com/talentdeficit/jsx) simply
>>> because it's not a NIF and thus doesn't impact the scheduler. But your
>>> needs and requirements may vary.
>>
>> It would be basically trivial to add {proplist()} support to
>> mochijson2 as an addition to {struct, proplist()}. There's not much of
>> a difference really, it's still wrapping a tuple around it but just
>> without a tag. I'll have a look at that.
>>
> An old version of Couchbeam accomplished that conversion thusly, if that
> helps:
>
[snip code]
Yeah, it was never hard to do it from the outside using object_hook on
decode and handler on encode. It's now natively supported in mochiweb
v2.1.0 without having to write any funs.
https://github.com/mochi/mochiweb
JSON = <<"{\"k\": \"v\"}">>.
[{<<"k">>, <<"v">>}] =:= mochijson2:decode(JSON, [{format, proplist}]).
{[{<<"k">>, <<"v">>}]} =:= mochijson2:decode(JSON, [{format, eep18}]).
%% the default
{struct, [{<<"k">>, <<"v">>}]} =:= mochijson2:decode(JSON, [{format, struct}]).
These will also round-trip with the encoder without any surprises or
additional options.
I've also added support for the {} and {P} format to kvc. kvc is
basically a simple query language for a host of proplist-like data
structures.
https://github.com/etrepum/kvc
-bob
_______________________________________________
erlang-questions mailing list
erlang-questions@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions
Post received from mailinglist |
|
|
| Back to top |
|
| Guest |
| Posted: Tue Aug 30, 2011 2:13 am |
|
|
|
Guest
|
On Mon, Aug 29, 2011 at 6:26 PM, Juan Jose Comellas <juanjo@comellas.org> wrote:
> May I suggest switching the decoded JSON format to that used by ejson
> (https://github.com/benoitc/ejson) and jiffy
> (https://github.com/davisp/jiffy) instead of the format used by
> mochiweb/mochijson2. Most of the projects that make heavy use of JSON seem
> to be moving to the format I'm suggesting.
By the way, there's another JSON library:
https://github.com/talentdeficit/jsx Somebody has asked me if I plan
to add support for it.
This is actually a very easy question. It takes less that 100 lines of
code to add support for any of the above libraries/decoded formats. If
somebody has a need for it and care to write the conversion code, I'll
be happy to merge it into Erlson and extend the existing API to allow
choosing a specific JSON codec.
For those who might be interested, adding XML support would not be
substantially harder.
Mochijson2 will probably remain the default JSON library in Erlson. It
is very stable, while statuses of the other libraries are not clear to
me. At least I couldn't figure this out by looking at their project
pages.
Anton
_______________________________________________
erlang-questions mailing list
erlang-questions@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions
Post received from mailinglist |
|
|
| Back to top |
|
|
|
All times are GMT
Page 1 of 2
Goto page 1, 2 Next
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum
|
|
|
Digg It
Del.icio.us
Reddit
Facebook
Stumble Upon
Technorati
