Big Data

118

Solutions

11

Content

i
The collection of articles, videos, and posts on this topic.

284

Contributors

Big data describes data sets that are so large or complex that traditional data processing applications are inadequate. Challenges include analysis, capture, data curation, search, sharing, storage, transfer, visualization, querying and information privacy. The term often refers simply to the use of predictive analytics or certain other advanced methods to extract value from data, and seldom to a particular size of data set.

Share tech news, updates, or what's on your mind.

Sign up to Post

hi,

any procedure and briefing on how to use Azure to implement big data in a cost efficient way ?

I am now studying how to setup big data cluster using SQL server 2019 on linux,  but I knew this is not enough.

and when connect to Azure SQL server, what tools should I use? SSMS 2019 or Azure data studio? any guide on that ?
0
9 Comments
I have a small "network" consisting of three computers and a "File Server" (peer to peer).  This supports an Access 216 database.  The server has a file that contains the "Back End" Tables, and each PC (Call them PCA, PCB, PCC) has an identical copy of the User Interface.  This has worked well for years.

PROBLEM:  All of a sudden some of the data displayed on each PC is different from data displayed on the others.  Example: Updating a record on one PC is NOT reflected on the other two [b/].  The same data table can have DIFFERENT NUMBER OF RECORDS on each PC[b/].  Worse:  If I open the Back End file on the "server" I find that in many cases the data there is also different from that on  some or all the PC's

I HAVE MADE VERY SURE THAT THERE IS ONLY ONE COPY OF THE DATA ON THE SERVER that is accessed by all three PC's

Example  The User Interface on PCA  can show 100 records for the widgets table and the User Interface on PCB may show 110 records while the User Interface on PCC could have either the same number (100 or 110) or it could have 95 records.

Is it possible that the file server has created separate copies of the same table.  I searched using a File Search program but could not fine any copies of the table(s).

Has anyone experienced something similar?  This is only happening since I installed Access 2016.  It never happened with Access 2003, 2007, 2010, 2013 etc.....

I look forward to hearing your comments.

Thanks,

Biggles1
0
6 Comments
I have installed  SQL Server 2017 on my Mac on Docker along with SQL Server Azure Data studio .  I have Cloudera virtual box installed on this Mac with Hadoop

Now I need to connect to my SQL Server 2017 and import a table into HDFC using SQOOP  . Can anyone please tell me what exactly I need to do to achieve this ?

Many Thanks
0
7 Comments
Hi Experts

Could you give me an overall strategie on how to mantain a centralized DB Server with its correspondnt databases receiving data from other decentrilized servers and databases from time to time?

The actualization between the decentrilized server and the centrilized server wouldn't need to be automatic, a D-1 actualization would be good (running when the resources comsumption is low).

The objective is to use Big Data analysis on the centralized DB.

Thanks in advance!
0
4 Comments
Please advice, best online self-paced big data paid course site
0
2 Comments
Having experience/Exposure to the below streams over 10 years
@ Administration-Linux-Unix/Oracle DB/Weblogic servers- Fusion Middleware

Kindly advise, Now moving to AWS platform, could leverage for current/future trend  ?
0
3 Comments
i have mvc / razor /c# app. It is reporting portal with ms sql server.
I use typically mvc template to built the app. and sometime, when I need to response back a big data using ajax call, it is just exceed the limit even i max up the webconfig.

can you show me some codes what i can do better? maybe I can zip down the size when ajax response come back to html in javascript or use json format at all time.
now my app response back to html string only.

Thanks
0
1 Comment
Kylin not getting started

here is the below error which I'm getting


kafka dependency is /opt/apache-kylin-2.2.0-bin/lib/kafka-clients-1.0.0.jar
Retrieving Spark dependency...
Error: Could not find or load main class exists
ERROR: Unknown error. Please check full log.
0
5 Comments
Hi I will like to know which could be the best DB open source for blockchain
0
2 Comments
Hive. How can I search for all tables in a database that contain a Column Name?
0
3 Comments
hi,

What is the support and feature on Data scientist and big data DB2 offer ?
0
5 Comments
hi,

anyone use polybase on MSSQL for hadoop ? is scale out feature of Polybase working fine? load balancing working well ?
0
16 Comments
hi,

for big data and data science solution of MS SQL, how many piece of puzzle we need? I know MS SQL has R service/server included in the MS SQL installation, I don't think only one server can do the job, what is the full picture?

each need separate cost ?
0
7 Comments
hi,

do oracle has graphical data model? so that for expensive operation like inner join, we can do it in graphical way so that inner join stuff can be by pass and use graphical process.

This is some time MS SQL use it for big data query.

Also will oracle trying to any kind of extra technology like GPU to speed up big data queries ?
0
9 Comments
hi,

What is the component/application for Oracle to do big data analysis?

What is the Oracle product for AI and data scientist research ?

when I look at this:

https://www.oracle.com/artificial-intelligence/platform.html

it said:

Additional libraries and tools include: Jupyter, pandas, scikit-learn, Pillow, OpenCV, and NumPy.
Deep learning frameworks include: TensorFlow, Keras and Caffe.
Elastic AI and Machine Learning Infrastructures include NVIDIA, Flash Storage, and Ethernet.

Open in new window

.

which component is free and no issue with Oracle DB. ?
0
14 Comments
hi,

any good reference , e.g. URL and books on designing , debugging and architecture on data lake with big data and data science?
0
5 Comments
I have some data I am trying to normalize/ weight. I have 4 regions, the number of people who have missing training certificates, and number of people in the region. Originally I was going to divide number of missing training certificate by number of people for each region to normalize the data. However, the data looks really small when I do that - like 100/40000. I don’t really want to graph such small numbers but need some way to bring in the number of people. Should I multiply by 100 and then just say this data is per 100 employees? Would that make sense?

Any other suggestions?
1
9 Comments
Hello Experts,

I would like to run a query on the attached file, but I don't know what type information is on the file in order to run a query on it.

Can someone let me know how to determine the information included in the file?

Regards

Carlton
VANQ_TRIAD_COLLS_20180118
0
2 Comments
Hello Community,

I have created my first hql code, see below and I can't get any data to appear.. I have recently installed Sandbox. The installation comes with a few sample databases. I'm using the database called sample_07 to guide me with my own .hql code.

My hql code is as follows:

CREATE EXTERNAL TABLE mysample
(
 code STRING,
 description STRING,
 total_emp INT,
 salary INT
)
ROW FORMAT DELIMITED
 FIELDS TERMINATED BY ','
STORED AS TEXTFILE
LOCATION '/root/music'
TBLPROPERTIES ("skip.header.line.count" = "1");

Open in new window


However, when I run the code using Zeppellin Notebook with the following code, I can see the tables, but no data appears

%jdbc(hive)
select * from mysample limit 14

Open in new window


However, when I run the same code, with using the sample database called sample_07 both the tables and data appear.

csharp

I'm sure there is something very simple that I'm missing.

Can someone please let me know where I'm going wrong?
0
3 Comments
I have be asked to move data dated before 2004.  Is there a easy way of doing this?  Without going through each folder, sorting by created date and then moving.
0
3 Comments
I'm interested in using Visual Studio in the field of BIG data and artificial intelligence.

At the moment the latest version of Visual Studio is 2017.
When is the next version due out?
What spec machine is needed for it to run smoothly in terms of processor, RAM and diskspace (and anything else that is relevant).
I found that with the Express edition, I could not use the Streamwriter.  Is this expected?
0
3 Comments
Hi,

what is the diff between MariaDB ColumnStore 1 0  and MS SQL SSIS + SSAS ? if MariaDB ColumnStore 1 0   ?
0
6 Comments
dear all,
I have got video and audio files I need to segment them based on their text.
I need to segment all the files. for example ( a single word contain n audio frames and n of visual frames (images) )
Can any one help or advice how can I make it?

Thanks
0
3 Comments
what is Big data hadoop ? how it works  and what software is required to run it ?
0
5 Comments
Hi experts, I'm having trouble sending file to .asp server using ajax as the code shown below. I've observed that too much character could hinder the asp server not to receive, sending an error says; "The source you are looking for has been removed, had its changed or is temporarily unavailable". My question is, Is there other way forcing the server to receive big data using Ajaz like the way i used below? What approach should i use to handle big data to send .asp server? Thanks experts!

  
function iGetPerona(x,code,inv)
{				
	if (window.XMLHttpRequest)
	  {// code for IE7+, Firefox, Chrome, Opera, Safari
	  xmlhttp = new XMLHttpRequest();
	  }
	else
	  {// code for IE6, IE5
	  xmlhttp=new ActiveXObject("Microsoft.XMLHTTP");
	  }
	xmlhttp.onreadystatechange=function()
	  {
	  if (this.readyState==4 && this.status==200)
	    {
	        var msg = this.response;
	        alert(msg);
	    }
	  }                                            
		xmlhttp.open("GET","SavePersona.asp?a="+x + "&idb="+ mcCode + "&b=" + code + "&c=" + inv, true);
		xmlhttp.send();
}

Open in new window
0
8 Comments

Big Data

118

Solutions

11

Content

i
The collection of articles, videos, and posts on this topic.

284

Contributors

Big data describes data sets that are so large or complex that traditional data processing applications are inadequate. Challenges include analysis, capture, data curation, search, sharing, storage, transfer, visualization, querying and information privacy. The term often refers simply to the use of predictive analytics or certain other advanced methods to extract value from data, and seldom to a particular size of data set.

Top Experts In
Big Data
<
Monthly
>
Hall of FameAnnual Expert Awards