[scala-user] Extending Graph for Scala for Datomic

Yes, I would define references in the schema to allow me to create
graphs. I plan to start tinkering with this idea by writing
loosely-coupled code to convert to and from the Datomic data model.
However, I think ultimately I will need a tight integration. That will
allow calls such as pathTo() can be performed against the DB, rather than
having to pull in the entire graph into memory.

This is my vision for the integration: I want to be able to write
functions that transform persistent graph data without using the Datomic
interface. I figure I'll still have a way to write Datalog queries to
retrieve a particular graph or set of data. As long as manipulating the
data is straight-forward, I'll say it's a success.

Joe

Hi Joe,

it's a good idea to start with a loosely-coupled preview which will prove
to be an easy task. You probably will have seen that JSON export/import and
Dot export, both separate modules within Graph for Scala, follow this
strategy.

Later on, you may also want to examine the ‘constrained’ module which
falls into the tightly-coupled category. It is a good example for how to
override behaviour at specific points by referring to extended
configuration parameters. While the configuration parameter of a
constrained graph contains its constraint, graphs reflecting persistent
data will have a configuration parameter providing the database connection,
schema description and alike.

As an aside, if you have specific restrictions on the types of connecting
nodes and their cardinalities, as it will be the case with Datomic schemas,
these restrictions could potentially be managed by a graph constraint in
the sense of the constrained module.
What about to depict the needed transformation process by writing a few
lines of test-first, non-compiling code right now? As I’m unsure about who
is interested to follow the details, we may also go off list with the
option to get back if somebody joins.

Peter


Am Samstag, 17. November 2012 23:43:06 UTC+1 schrieb Joe Barnes:

Yes, I would define references in the schema to allow me to create
graphs. I plan to start tinkering with this idea by writing
loosely-coupled code to convert to and from the Datomic data model.
However, I think ultimately I will need a tight integration. That will
allow calls such as pathTo() can be performed against the DB, rather than
having to pull in the entire graph into memory.

This is my vision for the integration: I want to be able to write
functions that transform persistent graph data without using the Datomic
interface. I figure I'll still have a way to write Datalog queries to
retrieve a particular graph or set of data. As long as manipulating the
data is straight-forward, I'll say it's a success.

Joe

Yes, I would define references in the schema to allow me to create
graphs. I plan to start tinkering with this idea by writing
loosely-coupled code to convert to and from the Datomic data model.
However, I think ultimately I will need a tight integration. That will
allow calls such as pathTo() can be performed against the DB, rather than
having to pull in the entire graph into memory.

This is my vision for the integration: I want to be able to write
functions that transform persistent graph data without using the Datomic
interface. I figure I'll still have a way to write Datalog queries to
retrieve a particular graph or set of data. As long as manipulating the
data is straight-forward, I'll say it's a success.

Joe

As for the schema factory methods, I arrived at this approach because I
wanted something like your predefined edge descriptors. Any hints on how
to better approach this are welcomed.

but I later began to think perhaps "scalax.collection.db.datomic" may be
preferable because of its shorter length.

On 17.11.2012 17:11, Joe Barnes wrote:
Hey Peter, thanks for your response. I recall reading in the
Graph for Scala documentation that you envision integration with
persistence libraries. That is partly what has inspired me to
consider trying this out.

There are several reasons why I'm more interested in Datomic than
Neo4j, both of which I have played around with. The most
compelling and unique feature of Datomic is the built-in history.
It never deletes data, so you can make queries against the past,
queries over deltas of time, etc. I can imagine lots of useful
applications of this powerful capability.

The second reason is that Datomic embraces the functional
paradigm by making all data immutable (hence the keeping of all
data). I believe it makes Datomic a better fit for Scala
persistence.

While I find the Neo4J concept very intriguing, I think the two
reasons I just gave are compelling enough for me to prefer
Datomic. Rather than thinking of a graph as the fundamental
relationship as in Neo4J, I would make a graph another data
structure that can be used, like a list is. (Just to drive the
point further home, we could likewise think of every field of an
object as a list, but with size == 1 as a special case) Since
Datomic supports recursive querying, it should be a good fit for
Graph for Scala.

Finally, I think a little background on me will be helpful too.
I'm a seasoned Java developer as I previously mentioned. I've
done a lot of work in Hibernate, so ORM has certainly poisoned my
thoughts on persistence. :) Scala and functional programming is
a very recent discovery for me. I consider myself more of a lazy
math geek that found out that computers will do the hard work for
me. I took graduate math courses in graph theory while earning
my Master's in CS. I would like to develop this Datomic
capability for me to use at my current employer. We're in the
telecom industry, and I need a robust way to store and reason
about network graphs. Because Datomic isn't free, there's still
a chance I'll have to fall back to Neo4J in the end.

Joe


On Saturday, 17 November 2012 03:52:11 UTC-6, Sonnenschein wrote:

Hi Joe,

this is great news. I don't think anybody has tried this.

Scala users have been expressing their need for making
Graph4Scala capable of reflecting graphs stored in graph
databases such as Neo4J but, I regret, I didn't find time to
implement such extensions and so far nobody has teamed up
with me. What I've already managed is a kind of preparation
for such extensions: whenever you instantiate a Graph there
is a seamless capability to provide configuration data. This
is essetially the similar to the recent extension of parallel
collections.

Could you tell us more about why you think Datomic
persistence, which is afaics object-based, as opposite to
graph-based, is still suited for a Graph4Scala facade?

Peter