Thanks Alex and Jorg. Frankly speaking I didn't think of details of
implementing such external DSL in ES. I am just from a user's perspective
thinking of what's the best user experience.

Jorg raises a good question that how to distinguish data type given the
nature of schemaless in ES, the DSL may have to have some keywords(e.g int,
long, date) to tell data type explicitly if it's important, otherwise all
data are treated as string. For other questions I can not answer because
some of features like facet I have not even used. I know it's not
straightforward to do such thing, but I do think if JSON DSL can solve
these problems then the external DSL can also do because it's more flexible
like a programming language. I have been playing with some NoSql databases
as well as search engine like ES, an impression I got is every software has
its own search DSL which are not user friendly. Though I think it might be
impossible for this area to agree on a standard similar to SQL given these
software focus on different parts and usually vary a lot, I think using a
SQL style DSL without join and sub-select can make user's life much easier.
On Friday, December 20, 2013 4:32:30 AM UTC-8, Jörg Prante wrote:

Just some food for thought:

In the DSL you give, there are some subtle issues which makes it hard for
implementing a parser and correct query execution.

For example, the word "New feature": is it just a phrase, or is it two
terms? Is a span query for the two terms also valid?

Another issue is well-formed input data typing. Is "ES" in the query an
input of type "string"? And is "3.1" is double value or also a string type?
Can types always be inferenced? What about date parsing, will date types be
assigned automatically? Maybe by looking up the ES mapping (well, ES
parsers do that already) ? Or, how to decide if "true" is the string "true"
or the boolean constant? ES catches this but I mean the general case of an
external DSL is dealing with JSON data type flaws. You are forced to
re-implement the ES parser for all these nasty pitfalls.

How to declare facets and filters in an external DSL? Or multi phrase and
multi wildcard searching? It's not straightforward if there is simply no
context information how to execute such things. How do you parse and
translate wildcards mixed/nested with phrases such as "scien* 'week*
magazin*'"? I was surprised how many folks are trained to use wildcards
excessively. It is only possible to replace such "bad queries" with
heuristics that can be executed on ES with high performance, with ranked
results etc.

I have written a CQL parser
http://docs.oasis-open.org/search-ws/searchRetrieve/v1.0/os/part5-cql/searchRetrieve-v1.0-os-part5-cql.pdfto generate ES DSL, but for Java API only. If anyone is interested in
adding a CQL parser as a REST action, I could offer it as a plugin. It is
of course not perfect, I'm not very satisfied with the result.

My experience so far is, at least for CQL, because it is a weak typed
query language (it does not have a notion of input data types) , that
external query languages must really be able to match the power of
Lucene/ES features, or you get into trouble implementing simplifications,
fallbacks, and shortcuts all the way.

So my favorite is still ES DSL, and for establishing simple searches,
there are special featured query types that are designed for simplified
free form input. E.g. ES DSL query type "query_string" understands the
Lucene syntax, or the "match" query, and now we also have the
"simple_query" https://github.com/elasticsearch/elasticsearch/pull/4402

I'm interested in OpenSearch for ES http://www.opensearch.org/ so if
anyone is working on this, it would be nice to know.

Jörg


On Thu, Dec 19, 2013 at 8:05 PM, xin zhang <xing...@gmail.com<javascript:>
wrote:


project = ES AND issuetype = "New Feature" AND fixVersion = 3.1 ORDER BY
created DESC, cf[10514] DESC

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0a5a07e4-666b-4498-a781-c48882fae2c6%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Search Discussions

Discussion Posts

Previous

Follow ups

Related Discussions

Discussion Navigation
viewthread | post
posts ‹ prev | 4 of 6 | next ›
Discussion Overview
groupelasticsearch @
categorieselasticsearch
postedDec 19, '13 at 7:05p
activeDec 21, '13 at 2:28p
posts6
users4
websiteelasticsearch.org
irc#elasticsearch

People

Translate

site design / logo © 2022 Grokbase