asked on

How do I fix this "$in needs an array?"

EE Ninjas!

We are poised on the threshold of great things, I just know it!

I'm working on a ticket that involves some pagination, but I have yet to be able to retrieve the list of "companies" that will utilize the pagination...

So, I've got to fix / understand a problem before I can fix the problem.

The good news is that I've been able to decipher where the error is coming from but I'm not sure what it means or how to fix it.

Given the fact that I'm still a pig on roller skates when it comes to NodeJs, I'm certain there's someone out there who can look at the error and know exactly what to do.

First of all, here's the code. This is "companies.js" and it's coming from the "services" directory.

datatable(input, baseFilters = {}) {
        return new Promise((resolve, reject) => {
            let config = this._getDataTablesFilters(input);
            let pipeline = [];
            let countPipeline = []; // the filters used for counts

            // add in the base filters
            config.filters = Util.merge(baseFilters, config.filters);

            // set the base pipeline where/match clause
            pipeline.push({ $match: this._enforceScope() });

            // add sorting
            if (Object.keys(config.sort).length > 0) {
                if (typeof config.sort[0] == 'undefined') {
                    pipeline.push({ $sort: config.sort });
                }

            }
            // Owner
            pipeline.push(
                {$lookup: {from: 'users',let: { 'ownerId': '$owner' },pipeline: [{$match: {$expr: {$and: [{ $eq: ["$_id", "$$ownerId"] }]}}},{$project: {name: {$concat: ['$firstName', ' ', '$lastName']}}}],as: 'owner'}},
                { $unwind: { path: '$owner', preserveNullAndEmptyArrays: true } }
            );
            // Broker            
            pipeline.push(
                { "$addFields": { "brokers": { "$slice": ["$brokers", 1] } } },
                { "$unwind": { path: "$brokers", preserveNullAndEmptyArrays: true } }
            );
            pipeline.push(
                { "$addFields": { "contacts": { "$slice": ["$contacts", 1] } } },
                { "$unwind": { path: "$contacts", preserveNullAndEmptyArrays: true } }
            );
            //pipeline.push(
            //    { "$addFields": { "addresses": { "$slice": ["$addresses", 1] } } },
            //    { "$unwind": { path: "$addresses", preserveNullAndEmptyArrays: true } }
            //);
             pipeline.push({
				$match: {
				  $or: [
					{ 'addresses.postal': { $in: global._user.account.postalCodes } },
					{ clientImport: global._user.account._id }
				  ]
				}
			  });


            // Search text      
            if (typeof input.search != 'undefined') {
                if (input.search.value) {

                    var searchQuery = this._getDataTablesSearch(input.search.value);
                    if (typeof searchQuery != 'undefined') {
                        pipeline.push({ $match: searchQuery });
                    }
                }
            }

            // set the filters pipeline where/match clause
            if (Object.keys(config.filters).length > 0) {
                pipeline.push({ $match: config.filters });
            }

            // set the count pipeline here since all of the filters should be applied
            // by this point
            countPipeline = Array.from(pipeline);




            // add the limits
            pipeline.push({ $skip: config.limits[0] });
            pipeline.push({ $limit: config.limits[1] });

            // add in a datatable specific field
            pipeline.push({
                $addFields: {
                    DT_RowId: '$_id',
                    currentUserId: global._user._id //for 'Assign to me' link
                }
            });

            // call the lifecycle method to allow extending classes to customize just
            // the pipeline without having to overwrite the whole method
            pipeline = this._beforeDatatablesPipeline(pipeline);


            this.model.aggregate(pipeline).then(async (data) => {
                resolve({
                    draw: parseInt(input.draw),
                    data: data,
                    recordsFiltered: await this.__aggCount(countPipeline),//filter wrong? await this._count(this._enforceScope()),
                    recordsTotal: await this._count(this._enforceScope())
                });
            }).catch((err) => {
				//console.log("it's here we've got a problem");
                console.log('Aggregate Error: ', err);
                reject(err);
            });
        });
    }

Open in new window

You can see where I've got "it's here where we've got a problem..." commented out so I'm pretty confident that this particular file is the piece of the puzzle that's going south.

Here's the error itself:

it's here we've got a problem
Aggregate Error:  MongoError: $in needs an array
    at Connection.<anonymous> (C:\wamp64\www\bSmart\node_modules\mongoose\node_modules\mongodb-core\lib\connection\pool.js:443:61)
    at Connection.emit (events.js:210:5)
    at processMessage (C:\wamp64\www\bSmart\node_modules\mongoose\node_modules\mongodb-core\lib\connection\connection.js:364:10)
    at TLSSocket.<anonymous> (C:\wamp64\www\bSmart\node_modules\mongoose\node_modules\mongodb-core\lib\connection\connection.js:533:15)
    at TLSSocket.emit (events.js:210:5)
    at addChunk (_stream_readable.js:309:12)
    at readableAddChunk (_stream_readable.js:290:11)
    at TLSSocket.Readable.push (_stream_readable.js:224:10)
    at TLSWrap.onStreamRead (internal/stream_base_commons.js:182:23) {
  operationTime: Timestamp { _bsontype: 'Timestamp', low_: 2, high_: 1584996241 },
  ok: 0,
  errmsg: '$in needs an array',
  code: 2,
  codeName: 'BadValue',
  '$clusterTime': {
    clusterTime: Timestamp { _bsontype: 'Timestamp', low_: 2, high_: 1584996241 },
    signature: { hash: [Binary], keyId: [Long] }
  },
  name: 'MongoError',
  [Symbol(mongoErrorContextSymbol)]: {}
}
(node:14708) UnhandledPromiseRejectionWarning: MongoError: $in needs an array
    at Connection.<anonymous> (C:\wamp64\www\bSmart\node_modules\mongoose\node_modules\mongodb-core\lib\connection\pool.js:443:61)
    at Connection.emit (events.js:210:5)
    at processMessage (C:\wamp64\www\bSmart\node_modules\mongoose\node_modules\mongodb-core\lib\connection\connection.js:364:10)
    at TLSSocket.<anonymous> (C:\wamp64\www\bSmart\node_modules\mongoose\node_modules\mongodb-core\lib\connection\connection.js:533:15)
    at TLSSocket.emit (events.js:210:5)
    at addChunk (_stream_readable.js:309:12)
    at readableAddChunk (_stream_readable.js:290:11)
    at TLSSocket.Readable.push (_stream_readable.js:224:10)
    at TLSWrap.onStreamRead (internal/stream_base_commons.js:182:23)
(node:14708) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:14708) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Open in new window

If my line of reasoning is sound, I'm thinking that the error is coming from line #103:

{ 'addresses.postal': { $in: global._user.account.postalCodes } },

But what does that mean?

One of the things that's baffling me is that I've looked all over for "postalCodes" in the database and I can't find anything, hence the puzzled look on my face.

What do you think?

SOLUTION

leakim971

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

Bruce Gust

ASKER

Sorry, Leak!

It's line #41, which you have already as far as "{ 'addresses.postal': { $in: [global._user.account.postalCodes] } },"

I kind of thought that was the case, but here's what made me trip:

I don't see "postalCodes" in any of these collections.

Do you have any ideas on where I could look that maybe I'm not seeing?

SOLUTION

Chris Stanyon

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

Michel Plungjan

There are some posts on the web about some issue appearing in v2.6 that was silent in 2.4, but it could be unrelated
https://jira.mongodb.org/browse/PHP-1051

ASKER CERTIFIED SOLUTION

leakim971

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

Bruce Gust

ASKER

Alright, guys...

I have spent the entire day popping the hood on this thing and while I'm not completely out of the woods, I do have some bottom lines.

The ticket I'm working on has to do with eliminating the pagination dynamic if there aren't enough rows to justify it. What you're seeing above is the error I was getting just to display the records.

So, I'm having to fix a problem... in order to fix a problem.

So, here's where I landed for the day:

It turns out that the error was coming from a mere "piece" of the page as opposed to a scenario where it was more of a comprehensive error.

the "/companies" route grabs some variables and sends them to the "companies.html.twig" file
part of that twig file includes a table that gets its data from yet another route...

<table id="main-dt" class="table custom make-datatable table-bordered table-striped dataTable no-footer" data-route="/companies/datatable" data-layout="tlipr" data-callback="initHeatMap();">

Open in new window

the "companies/datatable" route looks like this:

router.post('/companies/datatable', async (req, res) => {
    let resp = await Company.datatable(req.body, { pipeline: { $ne: '' } });
	console.log(resp);

    res.send(resp);
});

Open in new window

when you head out to that "service," you find a big method that has within it this little gem:

pipeline.push({
					$match: {
					  $or: [
						{ 'addresses.postal': { $in: global._user.account.postalCodes } },
						{ clientImport: global._user.account._id }
					  ]
					}
				  });

Open in new window

You smell that? That's my beast, right there!

Leaving it as is, this is what page looked like:

I inserted an IF clause...

if(global._user.account.postalCodes) {
				 pipeline.push({
					$match: {
					  $or: [
						{ 'addresses.postal': { $in: global._user.account.postalCodes } },
						{ clientImport: global._user.account._id }
					  ]
					}
				  });
			}

Open in new window

and I now I have this:

In other words, I made sure there was some data before I attempted to publish a table / chart.

Gentlemen...

This is huge!

Now, I'm not done. I still haven't nailed down where "postalCodes" is coming from, but in the name of staying focused on the ticket, I'm going to move forward and circle back later.

That said, I'm going to distribute points, close this question and then reference some other inquiries I'm now looking at, so if you're in the mood, feel free to weigh in yet again!

As always, thanks for your help!

Chris Stanyon

Good effort Bruce,

A huge part of being a successful developer is knowing how to dig through your code. The more you do it, the better you get :)

Michel Plungjan

Well done

How do I fix this &quot;$in needs an array?&quot;

How do I fix this "$in needs an array?"