FAQ
We are trying to migrate from our Map/Reduce implementation into a
new Aggregation Framework and are faced with the following
implementation detail:

Each of our documents contains two arrays of String values.

For example if a collection consists of a single document with the
following arrays:

{
arrayOne = ["A", "B", "C"],
arrayTwo = ["C", "D", "E", "F", "G"]
}

We need to sum up how many times the following combinations appear
where the first element is from the arrayOne and second element is
from arrayTwo:

["A", "D"],
["B", "E"]
["A", "Q"]

So from the above example

["A", "D"] should be counted for 1 as A appears in arrayOne and D
appears in arrayTwo,
["B", "E"] should be counted for 1 as B appears in arrayOne and E
appears in arrayTwo,
["A", "Q"] should be counted for 0 as A appears in arrayOne but Q does
not appear in arrayTwo

So, at the end of the the process, we need to get the sums for all the
documents we examined and come back with, something like:

["A", "D"] 25 times
["B", "E"] 100 times
["A", "Q"] 0 times

In our current Map/Reduce implementation we would emit("AD", 1) and
("BE", 1) and reduce would simply count those keys up. We can do
this, since there are JavaScript methods we can use to determine when
to emit these values.

I am not sure how to do the same in the aggregation framework, or is
this something that's beyond the aggregation framework means?

Thanks.
-AP_

--
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To post to this group, send email to mongodb-user@googlegroups.com.
To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.

Search Discussions

  • Siddharth Singh at Mar 21, 2012 at 3:12 pm
    I don't think you can do that with the aggregation framework.
    On Mar 20, 8:54 pm, Alex Paransky wrote:
    We are trying to migrate from our Map/Reduce implementation into  a
    new Aggregation Framework and are faced with the following
    implementation detail:

    Each of our documents contains two arrays of String values.

    For example if a collection consists of a single document with the
    following arrays:

    {
    arrayOne = ["A", "B", "C"],
    arrayTwo = ["C", "D", "E", "F", "G"]

    }

    We need to sum up how many times the following combinations appear
    where the first element is from the arrayOne and second element is
    from arrayTwo:

    ["A", "D"],
    ["B", "E"]
    ["A", "Q"]

    So from the above example

    ["A", "D"] should be counted for 1 as A appears in arrayOne and D
    appears in arrayTwo,
    ["B", "E"] should be counted for 1 as B appears in arrayOne and E
    appears in arrayTwo,
    ["A", "Q"] should be counted for 0 as A appears in arrayOne but Q does
    not appear in arrayTwo

    So, at the end of the the process, we need to get the sums for all the
    documents we examined and come back with, something like:

    ["A", "D"] 25 times
    ["B", "E"] 100 times
    ["A", "Q"] 0 times

    In our current Map/Reduce implementation we would emit("AD", 1) and
    ("BE", 1) and reduce would simply count those keys up.  We can do
    this, since there are JavaScript methods we can use to determine when
    to emit these values.

    I am not sure how to do the same in the aggregation framework, or is
    this something that's beyond the aggregation framework means?

    Thanks.
    -AP_
    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
  • Alex Paransky at Mar 21, 2012 at 3:58 pm
    I suppose this could be implemented "easier" with a simple count type
    of query. However, if I have a significant number of documents and
    many conditions that could get very slow in terms of performance.
    This is the reason we were using the Map/Reduce to begin with.

    So, is the basic application for the aggregation framework is to count
    instances of simple expressions (after all the unwinds have been
    done)? In other words, If you want to do something with a specific
    data point, virtual or real the aggregation framework will work
    nicely. However, if you want to aggregate a conditional expression
    such as (this and that but not this except when those in different
    graph paths) that may not be possible?

    I am just trying to figure out the general problem space that
    aggregation framework solves...

    Thanks.
    -AP_
    On Mar 21, 8:12 am, Siddharth Singh wrote:
    I don't think you can do that with the aggregation framework.

    On Mar 20, 8:54 pm, Alex Paransky wrote:






    We are trying to migrate from our Map/Reduce implementation into  a
    new Aggregation Framework and are faced with the following
    implementation detail:
    Each of our documents contains two arrays of String values.
    For example if a collection consists of a single document with the
    following arrays:
    {
    arrayOne = ["A", "B", "C"],
    arrayTwo = ["C", "D", "E", "F", "G"]
    }
    We need to sum up how many times the following combinations appear
    where the first element is from the arrayOne and second element is
    from arrayTwo:
    ["A", "D"],
    ["B", "E"]
    ["A", "Q"]
    So from the above example
    ["A", "D"] should be counted for 1 as A appears in arrayOne and D
    appears in arrayTwo,
    ["B", "E"] should be counted for 1 as B appears in arrayOne and E
    appears in arrayTwo,
    ["A", "Q"] should be counted for 0 as A appears in arrayOne but Q does
    not appear in arrayTwo
    So, at the end of the the process, we need to get the sums for all the
    documents we examined and come back with, something like:
    ["A", "D"] 25 times
    ["B", "E"] 100 times
    ["A", "Q"] 0 times
    In our current Map/Reduce implementation we would emit("AD", 1) and
    ("BE", 1) and reduce would simply count those keys up.  We can do
    this, since there are JavaScript methods we can use to determine when
    to emit these values.
    I am not sure how to do the same in the aggregation framework, or is
    this something that's beyond the aggregation framework means?
    Thanks.
    -AP_
    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.
  • Siddharth Singh at Mar 21, 2012 at 4:44 pm
    The new aggregation features are not a replacement for map-reduce. The
    idea behind aggregation frameworks is to make it possible to do a
    number of things much more easily, without having to resort to the big
    hammer that is map-reduce.

    Think of the kind of thing that SQL does with group-by and distinct.
    On Mar 21, 11:57 am, Alex Paransky wrote:
    I suppose this could be implemented "easier" with a simple count type
    of query.  However, if I have a significant number of documents and
    many conditions that could get very slow in terms of performance.
    This is the reason we were using the Map/Reduce to begin with.

    So, is the basic application for the aggregation framework is to count
    instances of simple expressions (after all the unwinds have been
    done)?  In other words, If you want to do something with a specific
    data point, virtual or real the aggregation framework will work
    nicely.  However, if you want to aggregate a conditional expression
    such as (this and that but not this except when those in different
    graph paths) that may not be possible?

    I am just trying to figure out the general problem space that
    aggregation framework solves...

    Thanks.
    -AP_

    On Mar 21, 8:12 am, Siddharth Singh wrote:






    I don't think you can do that with the aggregation framework.
    On Mar 20, 8:54 pm, Alex Paransky wrote:

    We are trying to migrate from our Map/Reduce implementation into  a
    new Aggregation Framework and are faced with the following
    implementation detail:
    Each of our documents contains two arrays of String values.
    For example if a collection consists of a single document with the
    following arrays:
    {
    arrayOne = ["A", "B", "C"],
    arrayTwo = ["C", "D", "E", "F", "G"]
    }
    We need to sum up how many times the following combinations appear
    where the first element is from the arrayOne and second element is
    from arrayTwo:
    ["A", "D"],
    ["B", "E"]
    ["A", "Q"]
    So from the above example
    ["A", "D"] should be counted for 1 as A appears in arrayOne and D
    appears in arrayTwo,
    ["B", "E"] should be counted for 1 as B appears in arrayOne and E
    appears in arrayTwo,
    ["A", "Q"] should be counted for 0 as A appears in arrayOne but Q does
    not appear in arrayTwo
    So, at the end of the the process, we need to get the sums for all the
    documents we examined and come back with, something like:
    ["A", "D"] 25 times
    ["B", "E"] 100 times
    ["A", "Q"] 0 times
    In our current Map/Reduce implementation we would emit("AD", 1) and
    ("BE", 1) and reduce would simply count those keys up.  We can do
    this, since there are JavaScript methods we can use to determine when
    to emit these values.
    I am not sure how to do the same in the aggregation framework, or is
    this something that's beyond the aggregation framework means?
    Thanks.
    -AP_
    --
    You received this message because you are subscribed to the Google Groups "mongodb-user" group.
    To post to this group, send email to mongodb-user@googlegroups.com.
    To unsubscribe from this group, send email to mongodb-user+unsubscribe@googlegroups.com.
    For more options, visit this group at http://groups.google.com/group/mongodb-user?hl=en.

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupmongodb-user @
categoriesmongodb
postedMar 21, '12 at 12:54a
activeMar 21, '12 at 4:44p
posts4
users2
websitemongodb.org
irc#mongodb

2 users in discussion

Alex Paransky: 2 posts Siddharth Singh: 2 posts

People

Translate

site design / logo © 2022 Grokbase