Avatar of Bruce Gust
Bruce Gust
Flag for United States of America asked on

Anybody ever heard of "leaflet.js" or "heatmap.js"?

Here's a screenshot of my page:

leaflet.js
Here's what you're looking at:

This particular company doesn't have any address information .Within the code, there's a piece that's looking for some relevant data based on the postal code. If that info doesn't exist, the page will just sit there until it finally times out leaving the user confused and bewildered.

To solve that dilemma, once I found the relevant syntax, I just qualified things by engaging that query only if there were some postal codes to query.

So, now for the first time in days, I'm actually able to look at some data on the page that, beforehand was simply timing out with no real explanation, at least as far as the user was concerned.

However...

As you can tell by the screenshot, while the page is firing, beneath the hood there's an error message that references "leaflet.js."

Upon closer inspection, "heatmap.js" is involved as well.

So, here's my dilemma...

Although there's only one row in my database at this point, there will most certainly be multiple rows, most of which will represent a complete array of information. In other words, they will have some postal codes.

But in those instances where you've got a renegade row with no postal codes, I want to prevent "heatmap" from firing.

Here's the twig file:

<table id="main-dt" class="table custom make-datatable table-bordered table-striped dataTable no-footer" data-route="/companies/datatable" data-layout="tlipr" data-callback="initHeatMap();">
      <thead>
        <tr>
          <th scope="col" data-field=":checkbox" data-searchable="0" data-sortable="0">
            <div class="pretty p-default">
                <input type="checkbox" class="table-all" />
                <div class="state p-primary">
                  <label>All</label>
                </div>
            </div>
          </th>
          <th scope="col" data-field="name" data-searchable="0" data-sortable="1">Company</th>
          <th scope="col" data-field="owner" data-searchable="0" data-sortable="1">Lead Owner</th>
            <th scope="col" data-field="contacts" data-searchable="0" data-sortable="1">Contacts</th>
          <th scope="col" data-field="addresses" data-searchable="0" data-sortable="1">Location</th>
          <th scope="col" data-field="meta.ftes" data-searchable="0" data-sortable="1">FTE's</th>
          <th scope="col" data-field="meta.premium" data-searchable="0" data-sortable="1">Premium</th>
          <!-- th scope="col" data-field=":actions">&nbsp;</th -->
        </tr>
      </thead>
      <tbody></tbody>
    </table>

Open in new window


I'm looking for a "for" loop and I don't see one.

How can I evaluate each hypothetical row and prevent any attempt on the part of the system to try and plot that on the map?

BTW: I'm more than happy to provide as much code as you want or need in order to comment intelligently, but I thought I would start with the basics first - especially since both heatmap.js is a third party thing and I'm hoping there's some ninjas who've worked with it before that can point me in the right direction.

Thanks!
Node.jsJavaScript

Avatar of undefined
Last Comment
Bruce Gust

8/22/2022 - Mon
SOLUTION
Julian Hansen

THIS SOLUTION ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
See Pricing Options
Start Free Trial
GET A PERSONALIZED SOLUTION
Ask your own question & get feedback from real experts
Find out why thousands trust the EE community with their toughest problems.
SOLUTION
lenamtl

THIS SOLUTION ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
See Pricing Options
Start Free Trial
GET A PERSONALIZED SOLUTION
Ask your own question & get feedback from real experts
Find out why thousands trust the EE community with their toughest problems.
Bruce Gust

ASKER
Fellas!

Sorry if I was vague, let me see if I can't better qualify my dilemma.

And while some of this may be superfluous, it's still a health exercise on my part to review the flow / logic of the code.

The route is "/companies." You've got a bunch of functionality that sets up the architecture of the page and then sends all of that to "companies.html.twig."

Part of that "twig" file, includes a section where you have a table that's defined like this:

<table id="main-dt" class="table custom make-datatable table-bordered table-striped dataTable no-footer" data-route="/companies/datatable" data-layout="tlipr" data-callback="initHeatMap();">

Open in new window


So, the content is coming from yet another route: "companies/datatable."

That route looks like this:

router.post('/companies/datatable', async (req, res) => {
    let resp = await Company.datatable(req.body, { pipeline: { $ne: '' } });
	//console.log(resp);
    res.send(resp);
});

Open in new window


When you head out to the "company" service to view the "datatable" method, you see this:

		//console.log(global._user.account.postalCodes);
		return new Promise((resolve, reject) => {
			let config = this._getDataTablesFilters(input);
			let pipeline = [];
			let countPipeline = []; // the filters used for counts

			// add in the base filters
			config.filters = Util.merge(baseFilters, config.filters);

			// set the base pipeline where/match clause
			pipeline.push({ $match: this._enforceScope() });

			// add sorting
			if (Object.keys(config.sort).length > 0) {
				if (typeof config.sort[0] == 'undefined') {
					pipeline.push({ $sort: config.sort });
				}

			}
			// Owner
			pipeline.push(
				{$lookup: {from: 'users',let: { 'ownerId': '$owner' },pipeline: [{$match: {$expr: {$and: [{ $eq: ["$_id", "$$ownerId"] }]}}},{$project: {name: {$concat: ['$firstName', ' ', '$lastName']}}}],as: 'owner'}},
				{ $unwind: { path: '$owner', preserveNullAndEmptyArrays: true } }
			);
			// Broker            
			pipeline.push(
				{ "$addFields": { "brokers": { "$slice": ["$brokers", 1] } } },
				{ "$unwind": { path: "$brokers", preserveNullAndEmptyArrays: true } }
			);
			pipeline.push(
				{ "$addFields": { "contacts": { "$slice": ["$contacts", 1] } } },
				{ "$unwind": { path: "$contacts", preserveNullAndEmptyArrays: true } }
			);
			//pipeline.push(
			//    { "$addFields": { "addresses": { "$slice": ["$addresses", 1] } } },
			//    { "$unwind": { path: "$addresses", preserveNullAndEmptyArrays: true } }
			//);
			if(global._user.account.postalCodes) {
				 pipeline.push({
					$match: {
					  $or: [
						{ 'addresses.postal': { $in: global._user.account.postalCodes } },
						{ clientImport: global._user.account._id }
					  ]
					}
				  });
			}

			// Search text      
			if (typeof input.search != 'undefined') {
				if (input.search.value) {

					var searchQuery = this._getDataTablesSearch(input.search.value);
					if (typeof searchQuery != 'undefined') {
						pipeline.push({ $match: searchQuery });
					}
				}
			}

			// set the filters pipeline where/match clause
			if (Object.keys(config.filters).length > 0) {
				pipeline.push({ $match: config.filters });
			}

			// set the count pipeline here since all of the filters should be applied
			// by this point
			countPipeline = Array.from(pipeline);

			// add the limits
			pipeline.push({ $skip: config.limits[0] });
			pipeline.push({ $limit: config.limits[1] });

			// add in a datatable specific field
			pipeline.push({
				$addFields: {
					DT_RowId: '$_id',
					currentUserId: global._user._id //for 'Assign to me' link
				}
			});

			// call the lifecycle method to allow extending classes to customize just
			// the pipeline without having to overwrite the whole method
			pipeline = this._beforeDatatablesPipeline(pipeline);

			this.model.aggregate(pipeline).then(async (data) => {
				resolve({
					draw: parseInt(input.draw),
					data: data,
					recordsFiltered: await this.__aggCount(countPipeline),//filter wrong? await this._count(this._enforceScope()),
					recordsTotal: await this._count(this._enforceScope())
				});
			}).catch((err) => {
				//console.log("it's here we've got a problem");
				console.log('Aggregate Error: ', err);
				reject(err);
			});
		});
    }

Open in new window


Look at line #39. That was my most recent "victory" because, in this instance, I've got a flawed row in that there isn't any "address" information. In that case, you've got a problem in that the page doesn't list anything. It just presents a "processing" graphic before it times out.

With that IF clause, the user is now able to see the information they're expecting, even if there isn't any postal code etc.

But that doesn't change the fact that the system still expects some kind of address data and rightfully so. But until that particular company is properly edited, the system is still attempting to retrieve geographical information and...

I can't see how it's doing that.

THAT is my question.

From what I can gather, the above code is passing an array into the table div.

Fine.

But how can I intercept that information in a way where, should there be any companies that don't have address data, the system isn't going to try and retrieve any "map" data?

I did see this in the "company.js" service file (line #1124):

const { ObjectId } = require('mongodb');
const { Util } = require('node-utils');
const moment = require('moment-timezone');
const mongoose = require('mongoose');
const uuid = require('uuid/v4');

const Service = require('./service');
const CompanySchema = require('../schemas/CompanySchema');
const ActivitySchema = require('../schemas/ActivitySchema');
const EventSchema = require('../schemas/EventSchema');
const ProposalSchema = require('../schemas/ProposalSchema');
const SavedFilterSchema = require('../schemas/SavedFilterSchema');
const UserSchema = require('../schemas/UserSchema');
const CompanyContactSchema = require('../schemas/CompanyContactSchema');
const CompanyAddressSchema = require('../schemas/CompanyAddressSchema');
const CompanyAddress = require('../services/companyaddress');
const GeoLocations = require('../services/GeoLocations');

const objectModels = require('../lib/object-models');
const Vendors = require('../lib/Vendors');

const ServiceSchema = require('../schemas/ServiceSchema');
const ProductsAndService = mongoose.model('Service', ServiceSchema);
const Proposal = mongoose.model('Proposal', ProposalSchema);
const User = require('../services/user.js');

let userModel = mongoose.model('User', UserSchema);

class Company extends Service {

    constructor() {
        super();

        this.collection = 'companies';
        this.plural = 'companies';
        this.singular = 'company';

        this.model = mongoose.model('Company', CompanySchema);

        this.fields = require('../lib/model-company');
        this.fields._id = {};
        this.fields._id.type = ObjectId;

        this.corePipeline = [
            {
                $lookup: {
                    from: 'users',
                    foreignField: '_id',
                    localField: 'owner',
                    as: 'owner'
                }
            },
            {$unwind: {path: '$owner',preserveNullAndEmptyArrays: true}},
                //{ "$addFields": { "brokers": { "$slice": ["$brokers", 1] } } },
                //{ "$unwind": { path: "$brokers", preserveNullAndEmptyArrays: true } },
                //{ "$addFields": { "contacts": { "$slice": ["$contacts", 1] } } },
                //{ "$unwind": { path: "$contacts", preserveNullAndEmptyArrays: true } },
                //{ '$match': { 'addresses.postal': { '$in': global._user.account.postalCodes } } }
            
        ];
    }

    datatable(input, baseFilters = {}) {
		//console.log(global._user.account.postalCodes);
		return new Promise((resolve, reject) => {
			let config = this._getDataTablesFilters(input);
			let pipeline = [];
			let countPipeline = []; // the filters used for counts

			// add in the base filters
			config.filters = Util.merge(baseFilters, config.filters);

			// set the base pipeline where/match clause
			pipeline.push({ $match: this._enforceScope() });

			// add sorting
			if (Object.keys(config.sort).length > 0) {
				if (typeof config.sort[0] == 'undefined') {
					pipeline.push({ $sort: config.sort });
				}

			}
			// Owner
			pipeline.push(
				{$lookup: {from: 'users',let: { 'ownerId': '$owner' },pipeline: [{$match: {$expr: {$and: [{ $eq: ["$_id", "$$ownerId"] }]}}},{$project: {name: {$concat: ['$firstName', ' ', '$lastName']}}}],as: 'owner'}},
				{ $unwind: { path: '$owner', preserveNullAndEmptyArrays: true } }
			);
			// Broker            
			pipeline.push(
				{ "$addFields": { "brokers": { "$slice": ["$brokers", 1] } } },
				{ "$unwind": { path: "$brokers", preserveNullAndEmptyArrays: true } }
			);
			pipeline.push(
				{ "$addFields": { "contacts": { "$slice": ["$contacts", 1] } } },
				{ "$unwind": { path: "$contacts", preserveNullAndEmptyArrays: true } }
			);
			//pipeline.push(
			//    { "$addFields": { "addresses": { "$slice": ["$addresses", 1] } } },
			//    { "$unwind": { path: "$addresses", preserveNullAndEmptyArrays: true } }
			//);
			if(global._user.account.postalCodes) {
				 pipeline.push({
					$match: {
					  $or: [
						{ 'addresses.postal': { $in: global._user.account.postalCodes } },
						{ clientImport: global._user.account._id }
					  ]
					}
				  });
			}


			// Search text      
			if (typeof input.search != 'undefined') {
				if (input.search.value) {

					var searchQuery = this._getDataTablesSearch(input.search.value);
					if (typeof searchQuery != 'undefined') {
						pipeline.push({ $match: searchQuery });
					}
				}
			}

			// set the filters pipeline where/match clause
			if (Object.keys(config.filters).length > 0) {
				pipeline.push({ $match: config.filters });
			}

			// set the count pipeline here since all of the filters should be applied
			// by this point
			countPipeline = Array.from(pipeline);




			// add the limits
			pipeline.push({ $skip: config.limits[0] });
			pipeline.push({ $limit: config.limits[1] });

			// add in a datatable specific field
			pipeline.push({
				$addFields: {
					DT_RowId: '$_id',
					currentUserId: global._user._id //for 'Assign to me' link
				}
			});

			// call the lifecycle method to allow extending classes to customize just
			// the pipeline without having to overwrite the whole method
			pipeline = this._beforeDatatablesPipeline(pipeline);


			this.model.aggregate(pipeline).then(async (data) => {
				resolve({
					draw: parseInt(input.draw),
					data: data,
					recordsFiltered: await this.__aggCount(countPipeline),//filter wrong? await this._count(this._enforceScope()),
					recordsTotal: await this._count(this._enforceScope())
				});
			}).catch((err) => {
				//console.log("it's here we've got a problem");
				console.log('Aggregate Error: ', err);
				reject(err);
			});
		});
    }


    getOne(companyId) {
        return new Promise((resolve, reject) => {
            companyId = this._getId(companyId);
            if (!companyId) {
                return reject('Invalid company ID provided.');
            }

            // create a clone of the pipeline so we don't alter the original
            let pipeline = this.corePipeline.slice(0);

            // add an initial match stage to the core pipeline
            pipeline.unshift({
                $match: this._enforceScope({
                    _id: companyId
                })
            });

            this.model.aggregate(pipeline).then((data) => {
                if (data.length === 0) {
                    reject('Could not find record.');
                } else {
                    resolve(data[0]);
                }
            }).catch((err) => {
                reject(err);
            });
        });
    }
    getOneContact(contactId, companyId) {
        return new Promise((resolve, reject) => {
            let company_Id = this._getId(companyId);
            let contact_id = this._getId(contactId);
            let pipeline = [{ $match: { _id: company_Id } },
            {$project: {
                contacts: {$filter: {
                    input: "$contacts",
                    as: "contacts",
                    cond: { $eq: ["$$contacts._id", contact_id] }
                    }}
            }},
            {$addFields:{'contacts.company':'$_id'}},
            {$unwind:{path:"$contacts",preserveNullAndEmptyArrays:true}},
                { $project: { 'contacts': 1, _id: 0 } }];
            this.model.aggregate(pipeline).then((rows) => {
                resolve(rows[0].contacts);
            }).catch((error) => {
                console.log(error.message);
                reject(error.message);
            });           
        });
    }
    saveContact(contactId, contact) {
        return new Promise((resolve, reject) => {
            let companyId = contact.company;
            contact.active = true;
            if (contactId === 'new') {                        
                delete contact.company;
                this.model.updateOne({ _id: companyId }, { $push: { contacts: contact } }, { setDefaultsOnInsert: true }).then(response => {                    
                    contact.company = companyId;
                    resolve(contact);
                }).catch(err => {
                    reject(err);
                });
            } else {
                this.model.updateOne({ _id: companyId },
                    {
                        $set: {
                            'contacts.$[i].email': contact.email,
                            'contacts.$[i].name': contact.name,
                            'contacts.$[i].phone': contact.phone
                        }
                    },
                    {
                        arrayFilters: [{
                                "i._id": this._getId(contactId)
                            }], setDefaultsOnInsert: true
                    }
                ).then(response => {
                    contact.company = companyId;
                    resolve(contact);
                }).catch(err => {
                    reject(err);
                });
            }
            
        });
    }
    getOneAddress(addressId, companyId) {
        return new Promise((resolve, reject) => {
            let company_Id = this._getId(companyId);
            let address_id = this._getId(addressId);
            let pipeline = [{ $match: { _id: company_Id } },
            {$project: {
                addresses: {$filter: {
                    input: "$addresses",
                    as: "addresses",
                    cond: { $eq: ["$$addresses._id", address_id] }
                    }}
            }},
            {$addFields:{'addresses.company':'$_id'}},
            {$unwind:{path:"$addresses",preserveNullAndEmptyArrays:true}},
                { $project: { 'addresses': 1, _id: 0 } }];
            this.model.aggregate(pipeline).then((rows) => {
                resolve(rows[0].addresses);
            }).catch((error) => {
                console.log(error.message);
                reject(error.message);
            });           
        });
    }
    saveAddress(addressId, address) {
        return new Promise((resolve, reject) => {
            let companyId = address.company;
            address.active = true;
            if (addressId === 'new') {
                delete address.company;
                this.model.updateOne({ _id: companyId }, { $push: { addresses: address } }, { setDefaultsOnInsert: true }).then(response => {
                    address.company = companyId;
                    resolve(address);
                }).catch(err => {
                    reject(err);
                });
            } else {
                this.model.updateOne({ _id: companyId },
                    {
                        $set: {
                            'addresses.$[i].street': address.street,
                            'addresses.$[i].city': address.city,
                            'addresses.$[i].state': address.state,
                            'addresses.$[i].postal': address.postal,
                            //'addresses.$[i].geo': geoaddress.geo,
                            'addresses.$[i].hash': address.hash,
                            'addresses.$[i].nonGeo': address.nonGeo
                        }
                    },
                    {
                        arrayFilters: [{
                            "i._id": this._getId(addressId)
                        }], setDefaultsOnInsert: true
                    }
                ).then(response => {
                    address.company = companyId;
                    resolve(address);
                }).catch(err => {
                    reject(err);
                });
            }

        });
    }
    getOneBroker(index, companyId) {
        return new Promise((resolve, reject) => {
            let company_Id = this._getId(companyId);
            this.model.findOne({ _id: company_Id, account: global._user.account._id }, (err, res) => {
                if (err) { console.log(err); reject(err); }
                resolve(res.brokers[index]);
            });
        });
    }

    saveBroker(index, broker) {
        return new Promise((resolve, reject) => {
            let companyId = this._getId(broker.company);
            let accountId = this._getId(broker.account);
            delete broker.company;
            delete broker.account;
            this.model.findOne({ _id: companyId, account: accountId }, (err, res) => {
                if (err) { console.log(err); reject(err); }
                res.brokers = res.brokers ? res.brokers : [];
                if (index == 'new') {                    
                    res.brokers.push(broker); 
                    broker = res.brokers;
                } else {
                    res.brokers[index] = broker;
                    broker = res.brokers;
                }
                this.model.updateOne({ _id: companyId, account: accountId }, { $set: { brokers: broker }}, (err, raw) => {
                    if (err) { console.log(err); reject(err); }
                    resolve(raw);
                });
            });
        });
    }
    //Fix: This function will check for dates field type array and convert it into object. Converts array field to Object
    resolveDatesArray() {
        return new Promise((resolve, reject) => {
            let condition = { dates: { $type: "array" } };
            this.model.updateMany(condition, { dates: {} }, { upsert: true }, (err, raw) => {
                if (err) { 
					console.log(err); 
					reject(err); 
				}
                resolve(raw);
            });            
        });
    }
    select2(query) {
        return new Promise((resolve, reject) => {
            this.model.aggregate([
                {
                    '$match': this._enforceScope({
                        name: {
                            '$regex': query,
                            '$options': 'i'
                        }
                    })
                },
                {
                    '$sort': {
                        name: 1
                    }
                },
                { '$limit': 25 },
                {
                    '$project': {
                        id: '$_id',
                        text: '$name'
                    }
                }
            ]).then((rows) => {
                resolve(rows);
            }).catch((err) => {
                reject(err);
            });
        });
    }

    assign(companies, user) {
        return new Promise((resolve, reject) => {
            let where = this._enforceScope({ _id: { $in: [] } });
            let update = { $set: {} };

            if (typeof user == 'undefined' || companies == 'undefined') {
                return reject('Invalid or missing parameters.');
            }

            let ids = [];
            for (let i in companies) {
                where._id['$in'].push(this._getId(companies[i]));
                ids.push(this._getId(companies[i]));
            }
            let pipeline = [{ $match: { _id: { $in: ids } } }, { $lookup: { from: "users", localField: "owner", foreignField: "_id", as: "owner" } }];
            this.model.aggregate(pipeline).then((beforeRows) => {


                update = {
                    $set: {
                        owner: this._getId(user),
                        account: global._user.account._id,//new added
                        active: true,//new added
                        'dates.assigned': new Date(moment().toISOString())// new added for reporting, pipeline target column display
                    }
                };                

                this.model.updateMany(where, update, (err, res) => {
                    if (err) {
                        reject(this.error('Invalid parameters provided.'));
                    } else {
                        let count = 0;
                        beforeRows.forEach((oldDoc) => {
                            pipeline = [{ $match: { _id: oldDoc._id } }, { $lookup: { from: "users", localField: "owner", foreignField: "_id", as: "owner" } }, { $limit: 1 }];
                            this.model.aggregate(pipeline).then(async (rows) => {
                                let newDoc = rows[0];
                                let meta = await this.jsonDiff(
                                    { owner: (oldDoc.owner.length > 0) ? '<a href="javascript:void(0);" class="user-info" data-user-id="' + oldDoc.owner[0]._id + '">' + oldDoc.owner[0].firstName + ' ' + oldDoc.owner[0].lastName + '</a>' : 'Unassigned' },
                                    { owner: (newDoc.owner.length > 0) ? '<a href="javascript:void(0);" class="user-info" data-user-id="' + newDoc.owner[0]._id + '">' + newDoc.owner[0].firstName + ' ' + newDoc.owner[0].lastName + '</a>' : 'Unassigned' }
                                );

                                let history = await this.addHistory('Owner change', newDoc._id, '', global._user._id, meta);
                                count = count + 1;
                                if (count = beforeRows.length) {
                                    resolve(this.success());
                                }
                            });

                        });
                    }
                });
            });
        });
    }

    unassign(companies) {
        return new Promise((resolve, reject) => {            
            let update = { $set: { owner: null } };
            let id;

            if (companies == 'undefined') {
                reject('Invalid or missing parameters.');
            }

            for (let i in companies) {
                id = this._getId(companies[i]);
                if (id) {
                    let pipeline = [
                        { $match: { _id: id } },
                        { $lookup: { from: "users", localField: "owner", foreignField: "_id", as: "owner" } },                        
                        { $limit: 1 }];
                    this.model.aggregate(pipeline).then((res) => {
                        if (res.length > 0) {                            
                            this.model.updateOne({ _id: id }, update, async(err, doc) => {
                                if (err) { console.log(err); }
                                if (typeof res[0].owner != 'undefined') {
                                    let meta = await this.jsonDiff(
                                        { owner: res[0].owner.length > 0 ? '<a href="javascript:void(0)" class="user-info" data-route="company" data-user-id="' + res[0].owner[0]._id + '">' + res[0].owner[0].firstName + " " + res[0].owner[0].lastName + '</a>' : "Unassigned" },
                                        { owner: 'Unassinged' }
                                    );
                                    await this.addHistory('Owner change', id, '', global._user._id, meta);
                                }                                
                                if ((parseInt(i)+1) === companies.length) {
                                    resolve(this.success());
                                }
                            });
                        }
                    }).catch((error) => { console.log(error); });
                }                
            }
        });
    }
    softDeleteItems(companies) {
        return new Promise((resolve, reject) => {
            if (typeof companies == 'undefined' || companies === 0) {
                return reject(Company.error('Please provide one or more company ID\'s.'));
            }            
            for (let i in companies) {
             let  id = this._getId(companies[i]);
                if (id) {
                    let pipeline = [
                        { $match: { _id: id } },
                        { $lookup: { from: "proposals", localField: "_id", foreignField: "company", as: "proposals" } },
                        { $limit: 1 }];
                    this.model.aggregate(pipeline).then((res) => {
                        if (res.length > 0) {
                            this.model.updateOne({ _id: id }, {$set: {deleted: true,active: false}}, async (err, doc) => {
                                if (err) { console.log(err); }
                                // delete Proposals
                                if (res[0].proposals.length > 0) {
                                    await Proposal.updateMany({ company: id }, { $set: { active: false } }, async (err, doc) => {
                                        if (err) { console.log(err);}
                                        // Todo nothing.
                                    });
                                }
                                let meta = await this.jsonDiff(
                                    { active: true },
                                    { active: false, deleted:true }
                                );
                                await this.addHistory('Company deleted', id, '', global._user._id, meta);

                                if ((parseInt(i) + 1) === companies.length) {
                                    resolve(this.success());
                                }
                            });
                        }
                    }).catch((error) => { console.log(error); });
                }
            }
        });
    }

    getActivity(companyId) {
        return new Promise((resolve, reject) => {
            const Activity = mongoose.model('Activity', ActivitySchema);
            //let pipeline = [
            //            {
            //                $match: {
            //                    account: global._user.account._id,
            //                    company: companyId
            //                }
            //            },
            //            {
            //                $sort: {
            //                    date: -1
            //                }
            //            },
            //            { $limit: 100 },
            //            {
            //                $lookup: {
            //                    from: 'users',
            //                    foreignField: '_id',
            //                    localField: 'user',
            //                    as: 'user'
            //                }
            //            },
            //            { $unwind: '$user' }
            //        ];
            //  Activity.aggregate(pipeline).then((rows) => {
            //  var activity = [];

            //  for (let i in rows) {
            //    // get the icon by type
            //    rows[i].icon = this.getIconByType(rows[i].type);

            //    // add it to the list
            //    activity.push(rows[i]);
            //  }

            //  resolve(activity);
            //}).catch((err) => {
            //  reject(err);
            //});
            let pipeline = [
                {
                    $match: {
                        account: global._user.account._id,
                        company: companyId
                    }
                },
                {
                    $lookup: {
                        from: 'users',
                        foreignField: '_id',
                        localField: 'user',
                        as: 'user'
                    }
                },
                { $unwind: '$user' },
                {
                    $group: {
                        '_id': {
                            year: { $year: "$date" },
                            month: { $month: "$date" },
                            day: { $dayOfMonth: "$date" }
                        },
                        activities: {
                            $push: {
                                _id: '$_id',
                                type: '$type',
                                company: '$company',
                                proposal: '$proposal',
                                user: { _id: '$user._id', 'firstName': '$user.firstName', 'lastName': '$user.lastName' },
                                date: '$date',
                                meta: '$meta'

                            }
                        }
                    }
                },
                { $project: { date: '$_id', activities: 1, _id: 0 } },
                { $sort: { "activities.date": -1 } }
            ];
            Activity.aggregate(pipeline).then((rows) => {
                var activity = [];

                for (let i in rows) {
                    for (let j in rows[i].activities) {
                        // get the icon by type
                        rows[i].activities[j].icon = this.getIconByType(rows[i].activities[j].type);                        
                        rows[i].activities[j].prettyDate = moment(rows[i].activities[j].date).fromNow();
                        rows[i].activities[j].readableDate = moment(rows[i].activities[j].date).format('MMM Do - LT');
                    }
                    // add it to the list 
                    rows[i].prettyDate = moment.utc(rows[i].date.year + '-' + rows[i].date.month + '-' + rows[i].date.day).local().fromNow();
                    activity.push(rows[i]);

                }

                resolve(activity);
            }).catch((err) => {
                reject(err);
            });
        });
    }

    getFiles(companyId) {
        return new Promise((resolve, reject) => {
            const Proposal = mongoose.model('Proposal', ProposalSchema);

            Proposal.aggregate([
                {
                    $match: {
                        company: companyId,
                        files: {
                            $exists: true
                        }
                    }
                },
                { $unwind: '$files' },
                {
                    $project: {
                        proposal: '$_id',
                        name: '$name',
                        file: '$files'
                    }
                }
            ]).then((rows) => {
                resolve(rows);
            }).catch((err) => {
                console.log('Error: ', err);
                reject(err);
            });
        });
    }

    addActivity(companyId, type, data) {
        return new Promise(async(resolve, reject) => {
            if (typeof type != 'string' || ['call', 'visit', 'reminder', 'event'].indexOf(type.trim()) == -1) {
                return reject('Invalid type provided');
            }
            const Schema = mongoose.Schema;
            const Base = mongoose.model('Activity', ActivitySchema);
            // to track activity on each pipeline stage
            if (data.proposal != 'undefined') {
                if (data.proposal != 'bulk') {
                    if (data.proposal) {
                        await Proposal.findById(this._getId(data.proposal), (err, res) => {
                            if (err) { console.log(err); }
                            if (res) {
                                data.pipeline = res.pipeline;
                            }
                        });
                    }
                }
            }
                       
            if (type == 'event') {
                // define dynamic Schema
                const Event = Base.discriminator('event', new mongoose.Schema({meta: {start: Date,end: Date,subject: String,participants: String,location: String,details: String}}));
                let _data = {
                    company: new ObjectId(companyId),
                    type: type.trim(),
                    meta: {
                        start: data.start,
                        end: data.end,
                        subject: data.subject,
                        participants: data.participants,
                        location: data.location,
                        details: data.details
                    },
                    user: global._user._id,
                    account: global._user.account._id
                };
                if (data.proposal != 'undefined') {
                    if (data.proposal != 'bulk') {
                        if (data.proposal) {
                            _data.proposal = data.proposal;
                            _data.pipeline = data.pipeline;
                        }
                    }
                }
                Event.create(_data, (err, record) => {
                    if (err) {
                        console.log('Error: ', err);
                        reject('An unexpected error was encountered. Please try again.');
                    } else {
                        resolve(record._id);
                    }
                });
            } else {
                // define dynamic Schema
                //FIX: descriminator with name 'call' already exists.
                const ActivityModel = Base.discriminators ? Object.keys(Base.discriminators).indexOf(type.trim()) >= 0 ? Base.discriminators[type.trim()] : Base.discriminator(type.trim(), new mongoose.Schema({ meta: { date: Date, notes: String } })) : Base.discriminator(type.trim(), new mongoose.Schema({ meta: { date: Date, notes: String } })); 
                //const ActivityModel = Base.discriminator(type.trim(), new mongoose.Schema({ meta: { date: Date, notes: String } })); 
                let _data = {
                    company: new ObjectId(companyId),
                    type: type.trim(),
                    meta: {
                        date: data.date,
                        notes: data.notes
                    },
                    user: global._user._id,
                    account: global._user.account._id
                };
                if (data.proposal != 'undefined') {
                    if (data.proposal != 'bulk') {
                        if (data.proposal) {
                            _data.proposal = data.proposal; 
                            _data.pipeline = data.pipeline;
                        }
                    }
                }
                ActivityModel.create(_data, (err, record) => {
                    if (err) {
                        console.log('Error: ', err);
                        reject('An unexpected error was encountered. Please try again.');
                    } else {
                        resolve(record._id);
                    }
                });
            }
        });
    }

    deleteActivity(activityId) {
        return new Promise((resolve, reject) => {
            let match = this._enforceScope({
                _id: new ObjectId(activityId)
            });

            const Activity = mongoose.model('Activity', ActivitySchema);

            Activity.deleteOne(match, (err) => {
                if (err) {
                    reject(err);
                } else {
                    resolve();
                }
            });
        });
    }

    exportToCSV(input) {
        return new Promise((resolve, reject) => {
            let config = this._getDataTablesFilters(input);
            let pipeline = [];
            let baseFilters = {
                account: global._user.account._id,
                active: true,
                deleted: {
                    $exists: false
                }
            };

            // add in the base filters
            config.filters = Util.merge(baseFilters, config.filters);

            // set the base pipeline where/match clause
            pipeline.push({ $match: this._enforceScope() });

            // lifecylce method to give the user the ability to inject setup stages to
            // the pipeline before we start adding in user-defined filters, limits and
            // sorting options
            pipeline = this._startPipeline(pipeline);

            // Search text      
            if (typeof input.search != 'undefined') {
                if (input.search.value) {

                    var searchQuery = this._getDataTablesSearch(input.search.value);
                    if (typeof searchQuery != 'undefined') {
                        pipeline.push({ $match: searchQuery });
                    }
                }
            }

            // set the filters pipeline where/match clause
            if (Object.keys(config.filters).length > 0) {
                pipeline.push({ $match: config.filters });
            }

            // add sorting
            if (Object.keys(config.sort).length > 0) {
                pipeline.push({ $sort: config.sort });
            }

            // we need to format the arrays - not sure what the heck is going on with
            // using $addresses.0.street in the projection stage below, but I had some
            // issues getting it to work
            pipeline.push({
                $addFields: {
                    _contact: {
                        $slice: ['$contacts', 1]
                    },
                    _address: {
                        $slice: ['$addresses', 1]
                    }
                }
            });

            pipeline.push({
                $unwind: {
                    path: '$owner',
                    preserveNullAndEmptyArrays: true
                }
            });

            pipeline.push({
                $unwind: {
                    path: '$_contact',
                    preserveNullAndEmptyArrays: true
                }
            });

            pipeline.push({
                $unwind: {
                    path: '$_address',
                    preserveNullAndEmptyArrays: true
                }
            });

            // add the project stage
            pipeline.push({
                $project: {
                    name: '$name',
                    owner: '$owner.name',
                    contact_name: '$_contact.name',
                    contact_email: '$_contact.email',
                    contact_phone: '$_contact.phone',
                    address_street: '$_address.street',
                    address_city: '$_address.city',
                    address_state: '$_address.state',
                    address_postal: '$_address.postal',
                    address_county: '$_address.county',
                    industry: '$meta.industry',
                    ftes: '$meta.ftes',
                    fteGrouping: '$meta.fteGrouping',
                    premium: '$meta.premium',
                    premiumPerFte: '$meta.premiumPerFte',
                    brokerCommissionTotal: '$meta.brokerCommissionTotal',
                    brokerFeesTotal: '$mata.brokerFeesTotal',
                    fundingType: '$meta.fundingType',
                    ein: '$meta.ein'
                }
            });

            // get the data
            this.model.aggregate(pipeline).then((rows) => {
                const papaparse = require('papaparse');
                console.log('Data: ', JSON.stringify(rows, undefined, 2));
                let csv = papaparse.unparse(rows);

                resolve(csv);
            }).catch((err) => {
                reject(err);
            });
        });
    }

    _startPipeline(pipeline) {
        let coreClone = this.corePipeline.slice(0);

        // remove the first stage of the core to add in the below projection
        coreClone.shift();

        // add in the owner's info
        coreClone.unshift({
            $lookup: {
                from: 'users',
                let: { 'ownerId': '$owner' },
                pipeline: [
                    {
                        $match: {
                            $expr: {
                                $and: [
                                    { $eq: ["$_id", "$$ownerId"] }
                                ]
                            }
                        }
                    },
                    {
                        $project: {
                            name: {
                                $concat: ['$firstName', ' ', '$lastName']
                            }
                        }
                    }
                ],
                as: 'owner'
            }
        });

        return pipeline.concat(coreClone);
    }
    targetSelected(companies = [], user) {
        return new Promise(async (resolve, reject) => {
            try {
                if (companies.length > 0 && user.length > 0) {
                    let Targetpipelines = await this.getPipelineColumns();                    
                    var index = Targetpipelines.map(function (o) { return o.key; }).indexOf("target");
                    let Targetpipeline = Targetpipelines[index];
                    if (typeof Targetpipeline != 'undefined') {
                        let count = 0;
                        companies.forEach((company) => {
                            let targetProposal = {
                                name: '_target', owner: this._getId(user), company: this._getId(company), pipeline: Targetpipeline._id, active: true, services: [], files: [],
                                //dates: { target: new Date(moment().toISOString())},
                                _range: [new Date(moment().toISOString()), new Date(moment().toISOString())],
                                _pipelineDate: new Date(),
                                pipelineIndex: 0,
                                account: global._user.account._id,
                                pipelineDuration: [{
                                    pipelineFrom: Targetpipeline._id, fDate: new Date(moment().toISOString()),
                                    pipelineTo: Targetpipeline._id, tDate: new Date(moment().toISOString())
                                }]
                            };
                            targetProposal.dates = typeof targetProposal.dates == 'undefined' ? {} : targetProposal.dates;
                            targetProposal.dates[Targetpipeline._id] = new Date(moment().toISOString()), new Date(moment().toISOString());
                            Proposal.updateOne({ owner: this._getId(user), company: this._getId(company), name: '_target' }, targetProposal, { runValidators: true, upsert: true }, async(error, res) => {
                                if (error) {
                                    console.log(error);
                                }
                                let targetUser = await this.getUserbyID(user);//await User.getOne(this._getId(user));
                                if (targetUser) {
                                    let meta = { 'Target Selected to': "<a href='javascript:void(0);' class='user-info' data-user-id='" + targetUser._id + "' data-route='market'>" + targetUser.firstName + " " + targetUser.lastName + "</a>" };
                                    this.addHistory('Target selected', this._getId(company), '', global._user._id, meta);
                                }
                                

                                count = count + 1;
                                if (count == companies.length) {
                                    resolve(true);
                                }
                            });
                        });
                    } else {
                        reject('Pipeline Column is undefined.');
                    }
                } else {
                    reject('Invalid or undefined company id or user id provided.');
                }
            } catch (e) {
                console.log(e);
                reject(e);
            }
        });
    }
    getUserbyID(id) {
        return new Promise(async (resolve, reject) => {
            if (id) {
                userModel.findById(this._getId(id), (err, res) => {
                    if (err) { console.log(err); resolve(false);}
                    resolve(res);
                });
                
            } else {
                resolve(false);
            }
        });
    }
    ignoreTargetSelected(companies = []) {
        return new Promise(async (resolve, reject) => {
            try {
                if (companies.length > 0) {
                    let Targetpipelines = await this.getPipelineColumns();
                    var index = Targetpipelines.map(function (o) { return o.key; }).indexOf("target");
                    let Targetpipeline = Targetpipelines[index];
                    if (typeof Targetpipeline != 'undefined') {
                        let companyIds = [];
                        companies.forEach(async (company) => {
                            companyIds.push(this._getId(company));
                            await this.addHistory('Target ignored', this._getId(company), '', global._user._id, {});
                        });
                        Proposal.deleteMany({ company: { $in: companyIds }, name: '_target' }, (error) => {
                            if (error) { reject(error); }
                            resolve(true);
                        });

                    } else {
                        reject('Pipeline Column is undefined.');
                    }
                } else {
                    reject('Invalid or undefined company id provided.');
                }
            } catch (e) {
                console.log(e);
                reject(e);
            }
        });
    }

    getOneForHistory(id) {
        return new Promise((resolve, reject) => {
            id = this._getId(id);
            if (!id) {
                return reject('Invalid prospect ID provided.');
            }
            let pipeline = [{ $match: { _id: id } },
            {
                $lookup: {
                    from: "users",
                    let: { "ownerId": "$owner" },
                    pipeline: [
                        { $match: { $expr: { $eq: ["$_id", "$$ownerId"] } } },
                        {
                            $project: {
                                _id: 0,
                                assigned: { $concat: ["<a href='javascript:void(0)' class='user-info' data-user-id='", { $toString: "$_id" }, "'>", "$firstName", " ", "$lastName", "</a>"] }
                            }
                        }
                    ],
                    as: "owner"
                }
            },
            { $unwind: { path: "$owner", preserveNullAndEmptyArrays: true } },
            { $addFields: { owner: "$owner.assigned" } }
            ];


            this.model.aggregate(pipeline).then((data) => {
                if (data.length === 0) {
                    reject('Could not find record.');
                } else {
                    resolve(data[0]);
                }
            }).catch((err) => {
                reject(err);
            });
        });
    }
    getProposals(companyId) {
        return new Promise((resolve, reject) => {
            let scopedFilters = this._enforceScope();
            var pipeline = [
                {
                    $match: {
                        company: this._getId(companyId),
                        name: { "$ne": "_target" },// should not show _target proposals they are just 'Target Selected' pipeline
                        $or: [{ "active": { "$exists": false } }, { "active": true }]
                    }
                },
                {
                    $lookup: {
                        from: "pipelinecolumns",
                        localField: "pipeline",
                        foreignField: "_id",
                        as: "pipeline"
                    }
                },
                { $unwind: "$pipeline" },
                {
                    $lookup: {
                        from: "services",
                        localField: "services.service",
                        foreignField: "_id",
                        as: "products"
                    }
                },
                {
                    $lookup: {
                        from: "users",
                        localField: "owner",
                        foreignField: "_id",
                        as: "owner"
                    }
                },
                {
                    $project: {
                        "products.model": 0, "products.account": 0, "products.amount": 0,
                        "owner.settings": 0, "owner.password": 0, "owner.email": 0, "owner.mobile": 0, "owner.account": 0
                    }
                },
                {
                    $unwind: "$owner"
                }];

            Proposal.aggregate(pipeline, (err, rows) => {
                if (err) {
                    reject(err);
                } else {
                    resolve(rows);
                }
            });
        });
    }
    getService(serviceId) {
        return new Promise((resolve, reject) => {
            let pipeline = [{ $match: { "_id": serviceId } }];
            ProductsAndService.aggregate(pipeline, (err, rows) => {
                if (err) {
                    reject(err);
                } else {
                    resolve(rows[0]);
                }
            });
        });
    }
    heatmap(ids) {
        return new Promise((resolve, reject) => {
            let pipeline = [];

            let _ids = [];
            if (ids && ids.length > 0) {
                ids.forEach((id) => {
                    _ids.push(this._getId(id));
                });

                let match = { $match: this._enforceScope() };
                delete match['$match']['account'];
                match['$match']._id = { $in: _ids };
                pipeline.push(match);
                pipeline.push(
                    { $unwind: { path: "$addresses", preserveNullAndEmptyArrays: true } },
                    { "$addFields": { "company": { _id: "$_id", name: "$name" } } },
                    {
                        "$lookup": {
                            "from": "geo_locations", "let": { "addressId": "$addresses._id" }, "pipeline": [
                                { "$match": { "$and": [{ "$expr": { "$eq": ["$address", "$$addressId"] } }, { "$expr": { "$eq": ["$active", true] } }] } }], "as": "geo_locations"
                        }
                    },
                    { $unwind: { path: "$geo_locations", preserveNullAndEmptyArrays: true } },
                    {
                        "$project": {
                            "_id": 0,
                            "type": "Feature",
                            "properties": {
                                "name": '$name',
                                "popupContent": "$name",
                                "_id": "$_id", "street": "$addresses.street", "city": "$addresses.city", 'state': '$addresses.state', "postal": "$addresses.postal", "country": "$addresses.country", "county": "$addresses.county",
                                "addresses": "$addresses",
                                "company": "$company", "nonGeo": "$addresses.nonGeo"
                            },
                            "geometry": "$geo_locations.geo"
                        }
                    }
                );
                this.model.aggregate(pipeline).then(async (rows) => {
                    let count = 0;
                    rows.forEach(async (row, index) => {
                        if (row.geometry === undefined) {
                            row.properties.addresses = row.properties.addresses ? row.properties.addresses : {};
                            row.properties.addresses.company = row.properties.company._id;
                            let geoAddress = await this.getGeoAddress(row.properties.addresses);
                            rows[index].geometry = geoAddress.geo;
                            count = count + 1;
                        } else { count = count + 1; }
                        if (rows.length == count) {
                            resolve({ "type": "FeatureCollection", "features": rows });
                        }
                    });
                }).catch((err) => {
                    console.log('Aggregate Error: ', err);
                    reject(err);
                });
            } else {
                //reject('undefined parameter ids provided.');
                resolve({ "type": "FeatureCollection", "features": [] });
            }

        });
    }

    getGeoAddress(address = {}) {
        return new Promise(async (resolve, reject) => {
            try {
                if (typeof address.geo != 'undefined') {
                    resolve(address);
                } else {
                    var isValidAddress = false;                    
                    address.street = address.street ? address.street.trim() : address.street;
                    if (!address || address.street === '' || address.city === '' || address.state === '' || address.postal === '' || address.street === undefined || address.city === undefined || address.state === undefined || address.postal === undefined) {
                        isValidAddress = false;
                        address.error = 'invalid';
                        resolve(address);
                    }
                    else {
                        isValidAddress = true;
                    }
                    if (isValidAddress) {
                        let geoaddress = await CompanyAddress.getGeocode(address);
                        if (geoaddress) {
                            if (geoaddress.error_message) {
                                address.error = geoaddress.error_message;
                                resolve(address);
                            } else {
                                GeoLocations.model.updateOne(
                                    { address: address._id },//, account: global._user.account._id
                                    { $set: { address: address._id, company: address.company, geo: geoaddress.geo } },//account: global._user.account._id,
                                    { upsert: true, setDefaultsOnInsert: true }, (err) => {
                                        if (err) {
                                            resolve(address);
                                        } else {
                                            this.model.updateOne({ _id: address.company },
                                                {
                                                    $set: {
                                                        'addresses.$[i].street': geoaddress.street,
                                                        'addresses.$[i].city': geoaddress.city,
                                                        'addresses.$[i].state': geoaddress.state,
                                                        'addresses.$[i].postal': geoaddress.postal,
                                                        //'addresses.$[i].geo': geoaddress.geo,
                                                        'addresses.$[i].hash': geoaddress.hash,
                                                        'addresses.$[i].nonGeo': geoaddress.nonGeo
                                                    }
                                                },
                                                {
                                                    arrayFilters: [
                                                        {
                                                            "i._id": address._id
                                                        }]
                                                }
                                            ).then(response => {
                                                resolve(geoaddress);
                                            }).catch(err => {
                                                address.error = err.msg;
                                                resolve(address);
                                            });
                                        }
                                    });

                                //CompanyAddress.model.updateOne({ _id: address._id }, { '$set': { street: geoaddress.street, city: geoaddress.city, state: geoaddress.state, postal: geoaddress.postal, geo: geoaddress.geo, hash: geoaddress.hash, nonGeo: geoaddress.nonGeo } }, { runValidators: true }, (err) => {
                                //    if (err) {
                                //        address.error = err.msg;
                                //        resolve(address);
                                //    } else {
                                //        resolve(geoaddress);
                                //        //let ret = { _id: location._id, name: record.name, address: location.address, coordinates: location.address.geo.coordinates };
                                //        //resolve(ret);
                                //    }
                                //});
                            }
                        } else {
                            address.error = 'invalid';
                            resolve(address);
                        }
                    }
                }
            } catch (e) {
                // return reject(e);
                console.log(e);//console.log(e.message);
                resolve(address);
            }
        });
    }
    

}

module.exports = new Company();

Open in new window


That looks pretty close, but I can't see how that might be connected.

So...

How can I qualify the data being assembled and whisked out the door by "datatables.js" in a way where, should there be an absence of address data, the system isn't making an attempt to retrieve geographical / map information?
ASKER CERTIFIED SOLUTION
leakim971

THIS SOLUTION ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
See Pricing Options
Start Free Trial
⚡ FREE TRIAL OFFER
Try out a week of full access for free.
Find out why thousands trust the EE community with their toughest problems.
Bruce Gust

ASKER
leak...

The problem with the app is that the user can add a company without adding the address in that the app assumes the user is going to add some contacts and it's only THEN that at any address data is being inputted.

I've alerted my boss and we're going to pause at this point and take an inventory if we don't want to go back and make that a required field.

In the interim, let me award some points and I appreciate your feedback!
Experts Exchange is like having an extremely knowledgeable team sitting and waiting for your call. Couldn't do my job half as well as I do without it!
James Murphy