Advertisement

11.12.2007 at 05:39PM PST, ID: 22956156 | Points: 500
[x]
Attachment Details

Combine and Internalize Three Queries in MySQL

Tags: mysql
Hi all - this may not be possible:

We'd like to run a query like the one below to find duplicate entries in our recordset, save those records to a separate table, and update the source table with a flag to mark the record as found.

Currently, we're doing all 3 outside access and one at a time:

Query 1) SELECT *
FROM MyTable AS mt
WHERE (((Exists (SELECT ColumnA, ColumnB, ColumnC
                         FROM MyTable AS a
                         WHERE  a.ColumnA = mt.ColumnA  
                         AND  ABS(a.ColumnB) = ABS(mt.ColumnB)  
                         AND  a.ColumnC = mt.Columnc    
                         GROUP BY a.ColumnA, a.ColumnB, a.ColumnC
                         HAVING Count(*)>1))<>False))
ORDER BY mt.ColumnA, mt.ColumnB, mt.ColumnC;

Query 2) loop through all records returned from Query 1 and insert those records in a new table, but process Query 3 below before moving on to next record from Query 1. There will be some blank/null values for ColumnG so I'm not sure what if/then language is available/needed.
INSERT INTO dup1b_report (
        MyTableID,
        ColumnA,
        ColumnB,
        ColumnC,
        ColumnD,
        ColumnE,
        ColumnF
        <if Len(ColumnG) GTE 1>, ColumnG</if>)
VALUES             (
        QUERY1.ID Value,
        QUERY1.ColumnA Value,
        QUERY1.ColumnB Value,
        QUERY1.ColumnC Value,
       ' QUERY1.ColumnD TextValue',
        QUERY1.ColumnE DateValue,
        QUERY1.ColumnF Value
        <if Len(ColumnG) GTE 1>, ColumnG Value</if>)

Query 3) Record that this loops record was identified in the original source table, then go on to the next record returned by Query 1.
UPDATE       MyTable
SET       MyTable.dup       =       1
WHERE      MyTable.ID      =      Query1.ID Value

Can MySQL do this all by itself? If so how? Can it handle running on large record sets? (10's and 100's of millions of rows) We also have a full copy of SQL Server 2005 standard if that is a better platform for handling this type of issue but we REALLY don't know that software.

Thanks for your help!
Start your free trial to view this solution
Question Stats
Zone: Database
Question Asked By: ecovarrubias
Question Asked On: 11.12.2007
Participating Experts: 1
Points: 500
Views: 0
Translate:
Loading Advertisement...
11.14.2007 at 03:09AM PST, ID: 20278957

Rank: Wizard

All comments and solutions are available to Premium Service Members only.

Start your 7 day free trial and see for yourself why Experts Exchange is the easiest and most proven technology resource in the world. Get Started

Already a member? Login to view this solution.

 
 
Loading Advertisement...
Microsoft
  • Internet Protocols
  • Applications
  • Development
  • OS
  • Hardware
  • Windows Security
Apple
  • Operating Systems
  • Hardware
  • Programming
  • Networking
  • Software
Internet
  • Search Engines
  • File Sharing
  • WebTrends / Stats
  • Spy / Ad Blockers
  • Web Browsers
  • New Net Users
  • Web Development
  • Chat / IM
  • Anti Spam
  • Web Servers
  • Anti-Virus
  • Email Clients
Gamers
  • Tips
  • Online / MMORPG
  • Puzzle
  • Emulators
  • Action / Adventure
  • Role Playing
  • Consoles
  • Game Programming
  • Strategy
  • Sports
  • Misc
  • Computer Games
Digital Living
  • Hardware
  • New Net Users
  • New Users
  • Software
  • Digital Music
  • Gaming World
  • Home Security
  • Apple
  • Networking Hardware
Virus & Spyware
  • Vulnerabilities
  • IDS
  • Encryption
  • Anti-Virus
  • Operating Systems Security
  • Software Firewalls
  • WebApplications
  • Cell Phones
  • Operating Systems
  • Internet
  • Hardware Firewalls
Hardware
  • Handhelds / PDAs
  • Displays / Monitors
  • Components
  • Networking Hardware
  • Peripherals
  • Laptops/Notebooks
  • Storage
  • Servers
  • Desktops
  • New Users
  • Misc
  • Apple
Software
  • System Utilities
  • Industry Specific
  • Network Management
  • Photos / Graphics
  • Page Layout
  • VMWare
  • Misc
  • Web Development
  • OS
  • CYGWIN
  • Voice Recognition
  • Message Queue
  • Quality Assurance
  • Security
  • Firewalls
  • MultiMedia Applications
  • Development
  • Database
  • Office / Productivity
  • Business Management
  • OS/2 Apps
  • Server Software
  • Internet / Email
ITPro
  • OS
  • Storage
  • Encryption
  • Operating Systems Security
  • Apple Hardware
  • Laptops & Notebooks
  • Servers
  • Networking Hardware
  • Peripherals
  • Devices
  • Displays / Monitors
  • WebTrends / Stats
  • Search Engines
  • Firewalls
  • WebApplications
  • IDS
  • Vulnerabilities
  • Email Clients
  • File Sharing
  • Spy / Ad Blockers
  • Web Browsers
  • Web Servers
  • Networking
  • Anti-Virus
  • Chat / IM
  • Anti Spam
Developer
  • Web Servers
  • Web Browsers
  • Game Programming
  • Dev Tools
  • Industry Specific
  • Office / Productivity
  • Database
  • CYGWIN
  • Web Development
  • Search Engines
  • File Sharing
  • WebTrends / Stats
  • Programming
  • Content Management
  • Application Servers
  • Protocols
Storage
  • Removable Backup Media
  • Storage Technology
  • Servers
  • Grid
  • Remote Access
  • Backup / Restore
  • Misc
  • Hard Drives
OS
  • Miscellaneous
  • Security
  • Development
  • Linux
  • VMWare
  • MainFrame OS
  • Unix
  • Apple
  • OS / 2
  • AS / 400
  • BeOS
  • Microsoft
  • VMS / OpenVMS
Database
  • Oracle
  • Miscellaneous
  • MySQL
  • Software
  • Sybase
  • Contact Management
  • PostgreSQL
  • Data Manipulation
  • Clarion
  • InterSystems Cache
  • Siebel
  • MUMPS
  • OLAP
  • SQLBase
  • SAS
  • GIS & GPS
  • 4GL
  • Berkeley DB
  • DB2
  • Informix
  • Interbase / Firebird
  • FoxPro
  • Reporting
  • LDAP
  • Filemaker Pro
  • MS SQL Server
  • dBase
  • MS Access
Security
  • Misc
  • Web Browsers
  • Software Firewalls
  • Operating Systems Security
  • File Sharing
  • Spy / Ad Blockers
  • Vulnerabilities
  • WebApplications
  • IDS
  • Anti-Virus
  • Encryption
  • Anti Spam
  • Email Clients
  • VPN
  • Chat / IM
Programming
  • Editors IDEs
  • Installation
  • Handhelds / PDAs
  • Multimedia Programming
  • System / Kernel
  • Algorithms
  • Game
  • Signal Processing
  • Project Management
  • Open Source
  • Database
  • Misc
  • Languages
  • Processor Platforms
  • Theory
Web Development
  • Scripting
  • Blogs
  • Web Servers
  • Software
  • Search Engines
  • Web Graphics
  • Images
  • Internet Marketing
  • Images and Photos
  • Components
  • Document Imaging
  • Web Languages/Standards
  • Illustration
  • WebApplications
  • Fonts
  • WebTrends / Stats
  • Authoring
  • Digital Camera Software
  • Miscellaneous
Networking
  • Protocols
  • Apple Networking
  • Network Management
  • Message Queue
  • Application Servers
  • Content Management
  • File Servers
  • Email Servers
  • Misc
  • Java Editors & IDEs
  • Wireless
  • Networking Hardware
  • Backup / Restore
  • System Utilities
  • ISPs & Hosting
  • Web Servers
  • Storage Technology
  • Removable Backup Media
  • Servers
  • Broadband
  • Grid
  • OS / 2
  • Novell Netware
  • Unix Networking
  • Windows Networking
  • Security
  • Telecommunications
  • Operating Systems
  • Linux Networking
Other
  • Community Advisor
  • Lounge
  • Community Support
  • New Net Users
  • Philosophy / Religion
  • Math / Science
  • Miscellaneous
  • URLs
  • Expert Lounge
  • Politics
  • Puzzles / Riddles
Community Support
  • Suggestions
  • New to EE
  • New Topics
  • Community Advisor
  • CleanUp
  • Announcements
  • General
  • Feedback
  • Input
  • EE Bugs
 
11.14.2007 at 03:09AM PST, ID: 20278957

Rank: Wizard

> Can MySQL do this all by itself? If so how?

The only way to automate it would be to put the code in a stored procedure and call it by a trigger. I don't think there an other way to do it...

>  Can it handle running on large record sets? (10's and 100's of millions of rows)

Yes. You'll need proper indexing and some server configuration for large databases and you should be fine.
 
 
20080236-EE-VQP-29 / EE_QW_2_20070628