MongoDB Query Processor - The Bottom Of The Flow Curve

Congratulations, this is the last new topic to learn about building a GraphQL server on NodeJS with a MongoDB backend! As mutations are quite similar to queries, you won't see much new after this.

Writing MongoDB queries is generally a somewhat specialized skill. There used to be a SQL Developer skillset - a person who knows SQL real well and at least one vendor-specific tool like PL/SQL. Not that MongoDB is so much easier that SQL. It's just that if your MongoDB queries become as complex as multi-page PL/SQL procedures - there is likely something fundamentally wrong with the platform selection and app design for your use case. You can do pretty complex things in MongoDB queries (as we briefly learned in the "MongoDB Datastore") chapter. You don't want to be doing a lot of complex things in them. It's not a logic running language - it's a helper tool. So, your average backend developer should be able to handle it well. No highly specialized skills and no dedicated team role for MongoDB queries.

Path to the Mongoose Code

Following the path from the demo listUsers GraphQL Query schema in src/relay-queries/user/user-list-query.js, which defines the resolver, through src/relay-resolvers/user-resolver.js, which defines the database query processor as queryFunction: fetchUserOrderList, we get to src/db-handlers/user/user-order-list-fetch.js, where we find

export const fetchUserOrderList = async (
  filterValues,
  aggregateArray,
  viewerLocale,
  fetchParameters
) => {

The list of parameters matches the logic coded in the pagination processor src/paging-processor/find-with-paging.js. To recap what was explained in the "Paging Flow Solution JavaScript Code" lesson, the following arguments are passed when calling the database query processing function from the pagination wrapper:

• filterValues is reserved for a free-form query string that we may be passing from the client; generally, not used in the demo
• aggregateArray is the sort/skip/limit component we apply to the query based on the client request and pagination logic implementation
• viewerLocale is an element of the viewer object that is frequently used when querying the database to extract language-specific subsets of data, e.g., en or fr. When coding your project, you may replace this with viewer or another element of it, based on the use case
• fetchParameters are assembled in the query resolver, usually, from the GraphQL query arguments

Inside fetchUserOrderList function, the logic is disappointingly simple. We use the straight-forward mongoose find method on the User model, with no conditions or projections to control which fields are included into the extract. Then we add the sort/skip/limit methods:

  let queryFunc = User.find();

  const sort = aggregateArray.find(item => !!item.$sort);
  if (sort) {
    queryFunc = queryFunc.sort(sort.$sort);
  }
  const skip = aggregateArray.find(item => !!item.$skip);
  if (skip) {
    queryFunc = queryFunc.skip(skip.$skip);
  }
  const limit = aggregateArray.find(item => !!item.$limit);
  if (limit) {
    queryFunc = queryFunc.limit(limit.$limit);
  }

The execution line is wrapped into try/catch:

  let userArray;
  try {
    userArray = await queryFunc.exec();
  } catch (err) {
    logger.error(`in fetchUserList exec: ` + err);
    throw new Error(`query failed`);
  }
  logger.debug(`  userArray ` + userArray);

The result of the query execution is returned to the paging wrapper to apply paging post-processing logic such as reference point cursor verification and cursor value generation for each record returned to the GraphQL client.

As an example, for the 4th page of users, sorted by full_name, the queryFunc will look like this:

User.find().sort({full_name: 1}).skip(29).limit(11)

The actual query evaluated from queryFunc and sent by mongoose to the MongoDB engine will be listed in the Mongoose debug logging output:

Mongoose: user.find({}, { sort: { full_name: 1, username: 1 }, skip: 29, limit: 11, projection: {} })

Error Handling

The try/catch behavior in JS is generally similar to that in Java: an error anywhere inside a chunk of code in the try part stops the execution of the chunk and triggers the catch part to be run. In NodeJS, however, it looks like the engine sometimes chokes up if an await line in the middle of a long try section throws and error or rejects the Promise. Rather than reaching the catch, the engine may complain about an unhandled promise rejection. Particularly, this may be happening if an await on a mongoose exec() fails. Considering that await is more of a syntactical way to pause the JS flow waiting for the result than the actual implementation of how async processes are handled by the JS engine, these side effects appear to be plausible. As a precaution, at least with the versions of NodeJS and mongoose used in the demo project, each mongoose await is wrapped in its own try/catch.

In the example above, we log the actual error so we can act on it in the log monitor, but we don't return error details to the client. This is the generally recommended practice protecting server internals from exposure to outside systems. In development, GraphQL is configured to run in the debug mode: in GraphiQL, we'll get a detailed error stack leading to the throw line. In production, the client will only receive the query failed message. The message can obviously be expanded to include some recommended steps, e.g., email address of the support team. In a multi-language system, the throw statement would be coded to return a message ID for the translation map vs. a verbal error message.

For a more controlled approach to GraphQL query error handling a designated query completion code field can be added to all GraphQL queries return objects. If an error was encountered - its value would be set accordingly. In such a design, all errors in the flow must be caught and an empty data reply with the error code set returned instead of throwing an error to the query resolver. The demo project uses this approach when processing Migrations. For Queries, the adopted design relies on the built-in error handling of the GraphQL engine that wraps errors into errors object of the response.

The behavior can be tested, e.g., by overriding the return userArray; line at the end of fetchUserOrderList with something like return new Error('test error')

The DB Query Response

In mongoose, data returned by a query can be an array of objects or a single object. If no matching records were found, either an empty JS array [] or a null is returned. Arguably, for consistency of coding, it would be reasonable to expect from mongoose producing uniform replies. Well, let's not bet on this to ever happen - think how much code out there has been written relying on the existing inconsistent output format. As a simple rule, findOne and findById return one record by design, or null if nothing was found, whereas find or aggregations return an array of objects or an empty array.

Looking through debug logging statement in the node-dev container console, this is what the database query response is when running listUsers:

 userArray 
  { primary_locale: 'en', _id: '1E4Yo11Y3r9a', full_name: 'Jane Doe', username: 'jane01',
  primary_email: 'jd@example.com', created_at: <date-time>Z, updated_at: <date-time>Z, __v: 0 },
  { primary_locale: 'en', _id: '1Eg6d1aFL8s7', full_name: 'John Public', username: 'john01',
  primary_email: 'jp@example.com', created_at: <date-time>Z, updated_at: <date-time>Z, __v: 0 }

__v: 0 is a special Document versioning field in mongoose. The demo project doesn't use it. The field is included into the database query output when there is no projection list specified. Never mind this field. mongoose documentation recommends keeping it in the schema (which is the default), so we do.

Rest of the Logic Flow: from DB Query to Output to the GraphQL Response

For each element of the database query output array, the GraphQL engine will execute the Object Type logic as per src/relay-models/user-type.js and populate records of the GraphQL response.

As a note, the GraphQL engine requires that the object returned from the query resolver corresponds to the expected plurality or singularity associated with the output Object Type. In this case, the Object Type is a Connection, so an array is expected. In getItem query we'll look at later in this chapter, a single object is expected. So, the fact that mongoose queries return either objects or arrays depending on which function is executed - works as an upside here.

The edge, node and cursor come from the logic coded in src/paging-processor/connection-from-datasource.js.

Lastly, you see this interfaces definition in src/relay-models/user-type.js:

interfaces: [NodeInterface]

An Interface is an abstract type in GraphQL that we don't cover in the course. When you define lots of Types in your application you may find Interfaces convenient to control which fields a group of similar Object Types that implement the Interface Type must have. The classic Animal-Cat-Dog example. However, the interfaces parameter here has a different meaning. It comes from graphql-relay nodeDefinitions. You remember that node = "the item at the end of the edge". nodeDefinitions is a helper object that you can find draft-coded in src/relay-models/node-definition-types.js

We don't maintain nodeDefinitions in the demo project, but it is included into graphql-relay Connections objects that we use. Per the graphql-relay design, nodeDefinitions should provide default methods of retrieving the content of GraphQL model objects from the underlying dataset using the object's GlobalID. We don't use this kind of stuff in the demo project. All queries have resolvers that implement the dedicated fetching. However, in a more simplistic design, e.g., when navigating over a Graph-like dataset, this direct retrieval of everything by its ID can actually work.


listUsers looks very simple to implement - and it is. Let's now look at a more complex query that lists User Orders with their Order Items. If you recall from previous chapters, Order Items will act as a sub-query, with its own paging. Sounds pretty difficult. Well, you'll be disappointed again - it is quite simple, border line primitive, to code. With JS and MongoDB - what else to expect?