NoSQL Databases

174

Solutions

326

Contributors

A NoSQL database provides a mechanism for storage and retrieval of data which is modeled in means other than the tabular relations used in relational databases. Motivations for this approach include: simplicity of design, simpler "horizontal" scaling to clusters of machines and finer control over availability. The data structures used by NoSQL databases (e.g. key-value, wide column, graph, or document) are specified from those used by default in relational databases, making some operations faster in NoSQL. Sometimes the data structures used by NoSQL databases are also viewed as "more flexible" than relational database tables.

Share tech news, updates, or what's on your mind.

Sign up to Post

I'm getting some error like this:

MongoError: E11000 duplicate key error collection: auth.users index: email_1 dup key: { email: "damianjnc114444455@gmail.com" }

Any ideas?

my code

https://github.com/damianjnc/server-authentication/tree/665d5bd2ac7e2b346d2616d5013311a5ff0cc430
0
Learn SQL Server Core 2016
LVL 13
Learn SQL Server Core 2016

This course will introduce you to SQL Server Core 2016, as well as teach you about SSMS, data tools, installation, server configuration, using Management Studio, and writing and executing queries.

We use SQL server 2017 as our main data store RDMS

We have a vehicle tracking app with one massive log table (all positions for vehicles)  and a lot of other small tables (like car number plate, users etc), pretty much every query involves a join to this main log table
We are just getting going, and the main table is over a billion rows and over 1TB with a relatively small number of cars in relation to the growth plans for the next year
I know this is small, but we estimate we will grow between 10x to 100x in the next year, which has brought up some architecture questions for long term.

Currently we use SQL server which runs nicely, we will also be adding redis to address reduce load on DB,
Currently the DB grows by 10GB every day
Considering we maybe growing at least by 100GB a day, over time this puts us into the BIG DATA category, and I need to start looking at some options for scaling
Every record that gets inserted to the log table gets an ID, and we use this ID for logging events, alerts etc for fast performance (this works well at the current sizing)

Currently my biggest headache is servers with disk storage, I can find server options upto 10TB quite easily, but after this options are limited and prices sky rocket.
Pricing is a massive issue, we are not Google/Microsoft and cannot afford huge server costs.
Performance is also a huge issue, for example, people expect to run reports and expect very small loading times.
We were also planning to move …
0
Hi
We have an existing application which tracks cars
We use a sql server DB to store the data

We have a new customer who has a report rate of every 5 seconds, and 300,000 devices
We plan to keep each 1 min of data in SQL server for 12 months, and the every 5 seconds of data in another data base for 72 hours to comply with regulations

The "other" database would be 1 table with the following fields
Imei Number
DateTime
Speed
Lat
Lon
IotDeviceRawData

it will only contain 72 hours of data, and will probably be terrabytes worth of data
It will be written to a lot, and queried infrequently (perhaps 10 times a day) to pull out data of a car

I'm open to options for different databases, it needs to be a open source DB (free) and fast to write to.
0
how to sequence or auto increment  in cassandra  from java /Spring code
0
Hello, I have built a .net solution which you can see here on github. The solution contains a web api project which is based on mongodb and uses easynetq. It also contains a console application which has a reference to the web api and also uses easynetq.
I was asked to deliver this solution wich a docker compose command, however I faced a few issues with the dockerfiles and the docker compose file, hence this answer. Whenever I add the Visual Studio support for Docker in my projects, then the solution fails to run....among other problems. Can you please provide me the correct docker files for this scenario?
0
A MongoDB automated script please that will :-

a) Backup MongoDB daily and using oplog to do point in-time restores if required.
b) Notify if backups have failed/success.
c) Remove backups after 7 days.

I would appreciate if you could do a sample script that will cover these areas as I am new to Mongo and would examples that would help me ?

Thanks
0
A script that will backup MongoDB:-

1. Backup the db at 00:01 (cron job..)
2. Compress the backup file
3. Remove backups after 2 days
4. Alert by email of backups successful or fail

A script that will be used to restore the MongoDB
0
Should I install memcache on a server which has MongoDB ?

What benefits will it bring ? Performance ?

Thanks
0
Assistance in backing up MongoDB databases:-

I am new to MongoDB and need some assistance in backing up databases by writing a script:-

a) Backup MongoDB at 00:01 kept for 7 days and are removed.  
b) An e-mail notification is sent to state that backups are completed or failed.
c) Is there a way to backup MongoDB i.e. transaction log backups or similar method, so 'point-in-time' restores can be done
0
Somebody has a good example autocomplete search nodejs, express, mongodb
0
CompTIA Security+
LVL 13
CompTIA Security+

Learn the essential functions of CompTIA Security+, which establishes the core knowledge required of any cybersecurity role and leads professionals into intermediate-level cybersecurity jobs.

mongodb SSL encryption with application makes slow.
Can somebody help on this
0
How to create select the following records via Mongo DB query

Assume database has following rows

Pen/TEST/PO1/
Pen/PSO/QCD-Formats/QCD-file.pdf
Pen/PSO/QOD-Formats/QOD-F-29(00)_Stage 2.doc -
Pen/PSO/PDF/SFO/1.txt
Pen/PSO/2.txt  
Pen/QOD-F-23(00)_Stage 2.doc

1st Query to fetch the exact record matching the given folder path

ex:
user gives Pen/TEST/PO1/

expected result
Pen/TEST/PO1/

assume user gives: Pen/PSO/QOD-Formats/QOD-F-29(00)_Stage 2.doc

expected result:
Pen/PSO/QOD-Formats/QOD-F-29(00)_Stage 2.doc

2nd Query to fetch all the records matching the given folder path and it's files alone and not sub-folder's files

assume user gives: Pen/PSO/

expected result

Pen/PSO/QCD-Formats
Pen/PSO/QOD-Formats
Pen/PSO/PDF
Pen/PSO/2.txt  

Thanks.
0
I am trying to connect to a MongoDB database with asp.net core in C# with a userid and password.  Can anyone assist me in that respect?
0
I have a system in elasticsearch collecting performance metrics from my network. The space occupied is a lot, and I need to know how to optimize disk space. I have the idea that the project consists of having detailed samples of the last time (for example, one or two weeks). Beyond that time you can have historical samples, but with less detail.
I need to start organizing how to carry this forward.
0
redis used as cache data in at some point if we want to persist to SQL/No SQL is there a straight forward way?
0
Sir,

I need a help on  Cassandra data model, is it a better way to design our table data structure   so that our bulk reads would be faster
please  suggest


CREATE TABLE IF NOT EXISTS screening_counts_by_location_year (
      total_screened_0_6 counter,
      male_0_6 counter,
      female_0_6 counter,

      total_screened_6_18 counter,
      male_6_18 counter,
      female_6_18 counter,

      total_screened counter,
      total_male counter,
      total_female counter,

      awc_screened counter,
      school_screened counter,
      college_screened counter,

      year varchar,

      location varchar,
      parent varchar,
      PRIMARY KEY((parent, location, year)));



CREATE TABLE IF NOT EXISTS screening_counts_by_location_year_mht (
      total_screened_0_6 counter,
      male_0_6 counter,
      female_0_6 counter,

      total_screened_6_18 counter,
      male_6_18 counter,
      female_6_18 counter,

      total_screened counter,
      total_male counter,
      total_female counter,

      awc_screened counter,
      school_screened counter,
      college_screened counter,
       mht_id varchar,
      year varchar,
      location varchar,
      parent varchar,
      PRIMARY KEY((parent, location, year), mht_id))
      WITH CLUSTERING ORDER BY (mht_id DESC);




CREATE TABLE rbsk_andr_v2.beneficiaryrecord (
    visitationpointtype text,
    reportdate date,
    zone text,
    district text,
    revenuedivision text,
    mandal text,
    village text,
    visitationpointcode text,
    universalid text,
    attendance text,
0
Dear Experts,

we have  back end application running on Cassandra and which has all our transaction data but we have a static data available in mysql
but i would like to know  if I would like to have queries between tables across two databases for the joins how that can be achieved.

Thanks
Roy...
0
Connection Error: No Mongos proxy available.

Environment: Node.js, MongoDB, Mongoose

Start.js
const mongoose = require('mongoose');


mongoose.connect(process.env.DATABASE,  { useNewUrlParser: true })
mongoose.Promise = global.Promise;
mongoose.connection
  .on('connected', () => {
    console.log(`Mongoose connection open on ${process.env.DATABASE}`);
  })
  .on('error', (err) => {
    console.log(`Connection error: ${err.message}`);
  });

.env file

DATABASE=mongodb://username:DBpassword@player-shard-00-00-1cmur.mongodb.net:27017,player-shard-00-01-1cmur.mongodb.net:27017,player-shard-00-02-1cmur.mongodb.net:27017
0
I want to write a init script to stop and start the MongoDB.
0
Bootstrap 4: Exploring New Features
LVL 13
Bootstrap 4: Exploring New Features

Learn how to use and navigate the new features included in Bootstrap 4, the most popular HTML, CSS, and JavaScript framework for developing responsive, mobile-first websites.

Hi,
i am currently working on the task of Data Migration from 1 cassandra cluster table to another cassandra cluster table using spark batch job.

i need a spark code for reading data from one cassandra cluster table and to process it in spark and then write results onto another existing cassandra cluster table.
0
{
 "_id" : "Sh",

"Name" : "HR",
"Form" : [
    {
        "Name" : "HR",
        "Permission" : {},
        "Fields" : [
            {
                "OutputFormat" : "Text",
                "Validation" : [],
                "Name" : "PASS",
                "Permission" : {
                    "Hidden" : [
                        "ac"
                    ]
                },
                "IsFormulaArg" : false,
                "MaxCharacters" : 100.0,
                "Label" : "PASS",
                "Widget" : "Dropdown",
                "DefaultPermission" : [],
                "Dropdown" : "me",
                "Score" : 1,
                "PermissionType" : "Hidden",
                "Id" : "me"
            },


            {
                "OutputFormat" : "dd/MM/yyyy",
                "NodeList" : {
                    "Score" : 2,
                    "Type" : "CellReference",
                    "Id" : "jdk2",
                    "Value" : null,
                    "CellMetadata" : {
                        "72uw" : "CreatedAt"
                    }
                },
                "Name" : "Date_Prepared",
                "Permission" : {},
                "IsFormulaArg" : false,
                "Required" : true,
                "Widget" : "Date",
                "Label" : "Date Prepared",
                "Score" : 2,
                "FormulaStr" : "CreatedAt",
                **"Formula" : "True",**
      …
0
So i have a user schema like this:

var user_schema = new Schema({
   username:{type:String,required: true, unique : true, trim: true},
   college:{type:String,required: true},
   password:{type:String,required: true, trim: true},
   email:{type:String,required: true, unique : true, trim: true},
   phone:{type:Number,required: true, unique : true, trim: true},
   dp:{type:String},
   tag:{type:String},
   description:{type:String},
   friends:[{type:String}],
   pending:[{type:String}],
   skills:{type:String},
   bucket:[{type:String}]
  });

Open in new window

and my objective is, to search the all the documents in the collection to get people based on the following conditions:

1. They should not be in the users' "friends" array.
2. They should not be in the users' "pending" array.
3. They should have the same "tag" (a string value) as the user.

So, basically I have to compare the users' fields ("friends","pending" and "tags"), with fields of all documents in the whole collection.

How do I do it, using mongoose (nodejs mongodb library)
0
I have to initialize an array to [](empty array) and then push user chose "tags" into it. This is my schema:

 var user_schema = new Schema({
   username:{type:String,required: true, unique : true, trim: true},
   college:{type:String,required: true},
   password:{type:String,required: true, trim: true},
   email:{type:String,required: true, unique : true, trim: true},
   phone:{type:Number,required: true, unique : true, trim: true},
   dp:{type:String},
   tags:[{type:String}],
   description:{type:String},
   skills:{type:String},
   bucket:[{type:String}]
 }); 

Open in new window

` and i have to initialize "tags" to [] and push user string values (user entered/chosen) every time the request is made to the API call.

I am using this line of code:

 stu_user.findOneAndUpdate(
   { _id: req.decoded.id },
   { $set:{tags:[]},$push: { tags: 'some chosen string'  } },
  function (error, success) {
         if (error) {
            console.log(error);
        } else {
            console.log(success);
        }
    });

Open in new window

But it's not working.
0
How to import collections from mongodb from cli?
0
Hey, This is just a decision-making problem. I am seeking for a well thought out answer.
I have to develop an android XMPP chat application, (the application also has a NodeJS server API, connecting to MongoDB and AWS S3 for picture uploads)

which will be better:

1. Having a "openfire" server on aws and connecting it to the android application and implementing an XMPP client on the android device using "smack" library.

2. Implementing a "xmpp-client" on NodeJS server side and scrape the results from this API to the Android device.
0

NoSQL Databases

174

Solutions

326

Contributors

A NoSQL database provides a mechanism for storage and retrieval of data which is modeled in means other than the tabular relations used in relational databases. Motivations for this approach include: simplicity of design, simpler "horizontal" scaling to clusters of machines and finer control over availability. The data structures used by NoSQL databases (e.g. key-value, wide column, graph, or document) are specified from those used by default in relational databases, making some operations faster in NoSQL. Sometimes the data structures used by NoSQL databases are also viewed as "more flexible" than relational database tables.