n1ql: How works?..is a total change in the "couchbase way"?

SQL++
1

Hi, I’m curious about n1ql and how it fits inside couchbase, my first thought was it used views from behind and was only a sugar for the “tradicional couchbase way” but it’s hasn’t totally sense because how couchbase would know how many views must create?..for example

SELECT name FROM contacts ORDER BY age LIMIT 3

this would require create a view where the key will be the name…when I use this query couchbase creates the view from behind and on the fly?..is this efficient?..

other example very interesting is using the sql AND statement inside n1ql…normally this would require create a collated view with arrays as keys [tomato,7] …in the n1ql this is transparent and pretty clear…couchbase create this view?..or this use other mechanism?

Other interesting stuff is than now you can access to a specific part in the json

SELECT children[0].fname AS cname
FROM tutorial
WHERE fname=‘Dave’

this mean than we can change the way how we create our “schemas”, normally is recommended separate the habitual accessed information from the not one

user1::data (not needed so frecuently)
user1::Articles (frecuenty accessed)

now…with the n1ql approach would be possible keep all the data inside the same document??..how this works??..

what about change information, would be possible not only retreived info and could change a specific field in the document, avoiding retreive all the document, change in the client side and send again?? (I’m pretty sure than it will not possible but I can’t resist the tentation of make the question :D)…

I feel than practically couchbase would be 2 db’s inside one (and it’s not about couchbdb and membase) with 2 totally different ways for working and if n1ql hasn’t limitations this means than the old way would become obsolete…the couchbase idea is than in the future we use n1ql abstraction and remove the actual “tricks” from handling data??

  1. Is possible use it with the actual sdks, for example the java sdk ??
  2. what about the performance?..

thanks so much and

thanks so much I really appreciate the effort you has placed in couchbase

2

Hello,

You ask a lot of interesting questions with good insights. I’ll try to answer all of them, and if I miss something, please let us know.

  • Yes, N1QL uses views, but does not create views on the fly. You can use CREATE INDEX and EXPLAIN to see how N1QL works.

  • N1QL has its own processing engine, which handles document / path traversal, expression evaluation, AND / OR and other logic, etc. N1QL uses views to identify documents, but then does its own processing to perform the query.

  • You can still design your buckets to optimize key-value throughput, e.g. separating fast-changing from slow-changing data, or small from large data. N1QL will include JOINs that will allow you to reassemble your data on the fly.

  • Yes, N1QL will include UPDATE … WHERE … statements for setting specific fields and paths without doing a round trip.

  • You will be able to use any Couchbase API or interface that you’d like.

  • Java SDK: See http://www.couchbase.com/communities/q-and-a/n1ql-java-sdk-api

  • N1QL performance depends on the underlying index, e.g. Couchbase views.

Thank you for your questions and feedback. There will be ongoing preview releases of N1QL with new features and improvements, followed by Beta and GA releases.

3

I am also curious about the internals of N1QL in terms of performance. I understand that each CREATE INDEX and CREATE PRIMARY INDEX will create a view to let the N1ql filter based on this query, but then all this views (at least from what I understood that couchbase does) comes from Disk (and maybe from disk cache if you are lucky). But then the rest of the process for filtering should be executed just on the hit server for that the query has been executed, isn’t it? Is there any discovery mechanism planned for N1QL servers from the client point of view (just thinking about redundancy and maybe load balancing)?

By the way, seems that you are making N1QL as a total separate product from Couchbase, any plans to integrate inside the core of couchbase to have some kind of extra performance? And in the other direction, like leave it as open that we can have in a future different index implementations (not just views)?

4

Robert,

Yes, N1QL will be integrated with the Couchbase server and cluster. The approach of a separate N1QL installation is only being used during Developer Preview, so as to make N1QL available without impacting the server. As to your other questions: yes, we are developing specialized indexes for N1QL; and N1QL queries work over the entire cluster transparently.

5

Hi Gerald,
You had give very good explaination of N1QL but I have some queries which are listed below:

  • First thing is that which is more efficient to use N1QL queries or view indexes?

  • Yo said that “N1QL will include UPDATE … WHERE … statements for setting specific fields and paths without doing a round trip.” My question is how it is possible to UPDATE…WHERE without doing a round trip?

  • You said that “N1QL performance depends on the underlying index, e.g. Couchbase views.” Here what does it meant by underlying index please ellaborate with example if possible.

Bundle of thanks in advance.

6

Hi @billalgill,

  • N1QL gives you more functionality and flexibility than map-reduce views. However, if you need a pre-computed aggregation, map-reduce views are faster.

  • With N1QL DML statements, there is no round-trip to the application / client.

  • Please see the documentation and blogs to understand indexing and to see examples. I would also recommend that you briefly look up SQL. N1QL is bringing many of the concepts of SQL and RDBMS (e.g. indexing) to NoSQL / Couchbase.

Thanks,
Gerald

1 Like
7

Thanks @geraldss for your explaination.