?
Solved

Get latest record in a file

Posted on 2009-04-23
14
Medium Priority
?
176 Views
Last Modified: 2013-12-12
I have the following code that reads a log file and inserts certain data (timestamp and number of users) from the log into a MySQL table.  This works fine.  However, the log is updated every 15 minutes and new data is appended to the bottom of the log. So I now need to just get  the data from the log where the timestamp is greater than $latest (i.e. the last entry in the database).  Any ideas?

The format for the log entries is:

<log4j:event logger="VistaMonitor" timestamp="1240375806540" time="Wed Apr 22 14:50:06 EST 2009" level="INFO" thread="VistaPooledThread-0:0" schema="LMS">
<log4j:message><![CDATA[

WebCT Session:  Current user count: 285

]]></log4j:message>
</log4j:event>

<log4j:event logger="VistaMonitor" timestamp="1240376106552" time="Wed Apr 22 14:55:06 EST 2009" level="INFO" thread="VistaPooledThread-0:0" schema="LMS">
<log4j:message><![CDATA[

WebCT Session:  Current user count: 280

]]></log4j:message>
</log4j:event>
<?php
require_once('connection/connect.php');
mysql_select_db($database, $connection);
 
$query = "SELECT * FROM node_a_data ORDER BY node_a_data_id DESC LIMIT 1";
$result = mysql_query($query, $connection) or die(mysql_error());
$row = mysql_fetch_assoc($result);
$latest = $row['timestamp'];
 
$log = "webct_monitor.log";
$openLog = fopen($log, 'r') or exit('Unable to open file!');
 
while(!feof($openLog)) {  
	$newline=fgets($openLog);
	if(strstr($newline, 'timestamp=')) {   
		$newline1 = explode('timestamp=', $newline);  
		$substring = explode('time=', $newline1[1]); 
		$timeStamp = str_replace('"','',$substring[0]);
	} elseif(strstr($newline, 'Current user count:')) {
		$substring = explode('Current user count:', $newline);  
		$userCount = $substring[1];  
		$query = "INSERT INTO node_a_data (node_a_data_id,timestamp,user_count) VALUES ('','$timeStamp','$userCount')";
		$result = mysql_query($query, $connection) or die(mysql_error());
	}  
}  
fclose($openLog);  
?>

Open in new window

0
Comment
Question by:peterbrowne
  • 8
  • 6
14 Comments
 
LVL 9

Expert Comment

by:richdiesal
ID: 24222054
Hi again peterbrowne.  This addition should do it for you.
<?php
require_once('connection/connect.php');
mysql_select_db($database, $connection);
 
$query = "SELECT * FROM node_a_data ORDER BY node_a_data_id DESC LIMIT 1";
$result = mysql_query($query, $connection) or die(mysql_error());
$row = mysql_fetch_assoc($result);
$latest = $row['timestamp'];
 
$log = "webct_monitor.log";
$openLog = fopen($log, 'r') or exit('Unable to open file!');
 
while(!feof($openLog)) {  
        $newline=fgets($openLog);
        if(strstr($newline, 'timestamp=')) {   
                $newline1 = explode('timestamp=', $newline);  
                $substring = explode('time=', $newline1[1]); 
                $timeStamp = str_replace('"','',$substring[0]);
        } elseif(strstr($newline, 'Current user count:')) {
                $substring = explode('Current user count:', $newline);  
                $userCount = $substring[1];  
                if ($timestamp > $latest) {
			$query = "INSERT INTO node_a_data (node_a_data_id,timestamp,user_count) VALUES ('','$timeStamp','$userCount')";
                	$result = mysql_query($query, $connection) or die(mysql_error());
		}
        }  
}  
fclose($openLog);  
?>

Open in new window

0
 
LVL 9

Expert Comment

by:richdiesal
ID: 24222073
Ahh... sorry, the new $timestamp should be $timeStamp.  It's always the little mistakes.
<?php
require_once('connection/connect.php');
mysql_select_db($database, $connection);
 
$query = "SELECT * FROM node_a_data ORDER BY node_a_data_id DESC LIMIT 1";
$result = mysql_query($query, $connection) or die(mysql_error());
$row = mysql_fetch_assoc($result);
$latest = $row['timestamp'];
 
$log = "webct_monitor.log";
$openLog = fopen($log, 'r') or exit('Unable to open file!');
 
while(!feof($openLog)) {  
        $newline=fgets($openLog);
        if(strstr($newline, 'timestamp=')) {   
                $newline1 = explode('timestamp=', $newline);  
                $substring = explode('time=', $newline1[1]); 
                $timeStamp = str_replace('"','',$substring[0]);
        } elseif(strstr($newline, 'Current user count:')) {
                $substring = explode('Current user count:', $newline);  
                $userCount = $substring[1];  
                if ($timeStamp > $latest) {
                        $query = "INSERT INTO node_a_data (node_a_data_id,timestamp,user_count) VALUES ('','$timeStamp','$userCount')";
                        $result = mysql_query($query, $connection) or die(mysql_error());
                }
        }  
}  
fclose($openLog);  
?>

Open in new window

0
 

Author Comment

by:peterbrowne
ID: 24222126
Thanks again for your help!

I did try exacty that before posting here, but while it sort of works, it duplicates the previous last entry as well as adding the latest entry:

(490, '1240376106560', 270),
(491, '1240376106560', 270), this was added
(492, '1240376106570', 50); then this
0
What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

 
LVL 9

Expert Comment

by:richdiesal
ID: 24222152
Hmm... that shouldn't happen.  When you run "SELECT * FROM node_a_data ORDER BY node_a_data_id DESC LIMIT 1"; do you get those values? --> (490, '1240376106560', 270)

Not necessarily through PHP - just a direct SQL query.
0
 
LVL 9

Expert Comment

by:richdiesal
ID: 24222165
Also, are you sure you used >?  The output you reported is what I'd expect if you'd used >=
0
 

Author Comment

by:peterbrowne
ID: 24222192
Yes to both questions.  The first query returns the last record in the table, which is of course the larger timestamp.  I am guessing that perhaps the code that reads the log should come as a result of a successful first query in order to get the sequence right.
<?php
require_once('connection/connect.php');
mysql_select_db($database, $connection);
 
$query = "SELECT * FROM node_a_data ORDER BY node_a_data_id DESC LIMIT 1";
$result = mysql_query($query, $connection) or die(mysql_error());
$row = mysql_fetch_assoc($result);
$latest = $row['timestamp'];
 
$log = "webct_monitor.log";
$openLog = fopen($log, 'r') or exit('Unable to open file!');
 
while(!feof($openLog)) {  
	$newline=fgets($openLog);
	if(strstr($newline, 'timestamp=')) {   
		$newline1 = explode('timestamp=', $newline);  
		$substring = explode('time=', $newline1[1]); 
		$timeStamp = str_replace('"','',$substring[0]);
        } elseif(strstr($newline, 'Current user count:')) {
                $substring = explode('Current user count:', $newline);  
                $userCount = $substring[1];  
                if ($timeStamp > $latest) {
                        $query = "INSERT INTO node_a_data (node_a_data_id,timestamp,user_count) VALUES ('','$timeStamp','$userCount')";
                        $result = mysql_query($query, $connection) or die(mysql_error());
                }
        }  
}  
fclose($openLog);  
?>

Open in new window

0
 
LVL 9

Expert Comment

by:richdiesal
ID: 24222233
Well, what that code should do is produce the same data as if it were INSERTing every line, but only insert it if the $timeStamp was larger $latest.  Which means the code output doesn't make sense.

I'm curious - can you change it to >= and post what the first 3 or 4 lines of that output looks like?
0
 

Author Comment

by:peterbrowne
ID: 24222356
I am going to have to go home in a minute, but here are the last 3 entries after running the script on an empty table:

(491, '1240376106580', 50),
(492, '1240376106590', 15),
(493, '1240376106600', 49);

I then deleted record 493 and ran the script again and 2 records were created (494 and 495).  This is with the >= and is the same with > (I know it seems odd...)

(492, '1240376106590', 15),
(494, '1240376106590', 15),
(495, '1240376106600', 49);

I have added the table sql and the webct_monitor.log (if you use it just change the extension to .log).  I will have a look to see if you have a solution tomorrow.  Thanks for your help!



-- phpMyAdmin SQL Dump
-- version 2.11.9.2
-- http://www.phpmyadmin.net
--
-- Host: localhost
-- Generation Time: Apr 24, 2009 at 04:46 PM
-- Server version: 4.1.22
-- PHP Version: 4.3.9
 
SET SQL_MODE="NO_AUTO_VALUE_ON_ZERO";
 
--
-- Database: `cedata`
--
 
-- --------------------------------------------------------
 
--
-- Table structure for table `node_a_data`
--
 
CREATE TABLE IF NOT EXISTS `node_a_data` (
  `node_a_data_id` int(5) NOT NULL auto_increment,
  `timestamp` varchar(15) NOT NULL default '',
  `user_count` int(5) NOT NULL default '0',
  PRIMARY KEY  (`node_a_data_id`)
) ENGINE=MyISAM  DEFAULT CHARSET=latin1 AUTO_INCREMENT=496 ;
 
--
-- Dumping data for table `node_a_data`
--
 
INSERT INTO `node_a_data` (`node_a_data_id`, `timestamp`, `user_count`) VALUES
(1, '1240230900533', 255),
(2, '1240231200608', 285),
(3, '1240231500534', 319),
(4, '1240231800550', 343),
(5, '1240232100544', 370),
(6, '1240232400537', 390),
(7, '1240232700545', 393),
(8, '1240233000535', 379),
(9, '1240233300551', 359),
(10, '1240233600558', 355),
(11, '1240233900539', 332),
(12, '1240234200526', 312),
(13, '1240234500527', 298),
(14, '1240234800543', 288),
(15, '1240235100533', 293),
(16, '1240235400537', 284),
(17, '1240235700541', 273),
(18, '1240236000531', 264),
(19, '1240236300535', 251),
(20, '1240236600534', 245),
(21, '1240236900530', 224),
(22, '1240237200536', 216),
(23, '1240237500531', 205),
(24, '1240237800524', 194),
(25, '1240238100516', 183),
(26, '1240238400523', 164),
(27, '1240238700537', 151),
(28, '1240239000531', 145),
(29, '1240239300534', 134),
(30, '1240239600360', 124),
(31, '1240239900355', 121),
(32, '1240240200367', 115),
(33, '1240240500371', 113),
(34, '1240240800371', 109),
(35, '1240241100377', 97),
(36, '1240241400377', 97),
(37, '1240241700408', 89),
(38, '1240242000398', 92),
(39, '1240242300390', 91),
(40, '1240242600403', 85),
(41, '1240242900419', 71),
(42, '1240243200359', 70),
(43, '1240243500359', 70),
(44, '1240243800361', 65),
(45, '1240244100365', 61),
(46, '1240244400382', 58),
(47, '1240244700373', 56),
(48, '1240245000384', 49),
(49, '1240245300395', 49),
(50, '1240245600399', 50),
(51, '1240245900395', 43),
(52, '1240246200408', 36),
(53, '1240246500424', 29),
(54, '1240246800360', 29),
(55, '1240247100365', 28),
(56, '1240247400367', 24),
(57, '1240247700375', 24),
(58, '1240248000388', 20),
(59, '1240248300381', 20),
(60, '1240248600384', 20),
(61, '1240248900388', 21),
(62, '1240249200394', 18),
(63, '1240249500409', 17),
(64, '1240249800418', 17),
(65, '1240250100411', 17),
(66, '1240250400379', 17),
(67, '1240250700387', 18),
(68, '1240251000372', 18),
(69, '1240251300386', 17),
(70, '1240251600393', 17),
(71, '1240251900384', 17),
(72, '1240252201312', 16),
(73, '1240252500401', 16),
(74, '1240252800410', 15),
(75, '1240253100407', 16),
(76, '1240253400409', 13),
(77, '1240253700421', 11),
(78, '1240254000376', 8),
(79, '1240254300375', 8),
(80, '1240254600390', 8),
(81, '1240254900434', 7),
(82, '1240255200393', 7),
(83, '1240255500393', 6),
(84, '1240255800307', 6),
(85, '1240256100314', 7),
(86, '1240256400325', 11),
(87, '1240256700322', 11),
(88, '1240257000330', 12),
(89, '1240257300337', 15),
(90, '1240257600343', 19),
(91, '1240257900344', 20),
(92, '1240258200352', 22),
(93, '1240258500353', 24),
(94, '1240258800360', 25),
(95, '1240259100365', 31),
(96, '1240259400328', 32),
(97, '1240259700330', 34),
(98, '1240260000333', 42),
(99, '1240260300346', 44),
(100, '1240260600357', 45),
(101, '1240260900339', 46),
(102, '1240261200360', 51),
(103, '1240261500352', 55),
(104, '1240261800357', 50),
(105, '1240262100369', 58),
(106, '1240262400366', 69),
(107, '1240262700376', 75),
(108, '1240263000331', 80),
(109, '1240263300338', 78),
(110, '1240263600334', 85),
(111, '1240263900342', 89),
(112, '1240264200366', 104),
(113, '1240264500364', 110),
(114, '1240264800364', 118),
(115, '1240265100386', 120),
(116, '1240265400380', 124),
(117, '1240265700390', 127),
(118, '1240266000402', 145),
(119, '1240266300384', 158),
(120, '1240266600338', 173),
(121, '1240266900404', 194),
(122, '1240267200383', 207),
(123, '1240267500364', 233),
(124, '1240267800397', 240),
(125, '1240268100421', 261),
(126, '1240268400390', 265),
(127, '1240268700411', 273),
(128, '1240269000419', 280),
(129, '1240269300397', 281),
(130, '1240269600397', 284),
(131, '1240269900402', 285),
(132, '1240270200364', 288),
(133, '1240270500367', 278),
(134, '1240270800372', 286),
(135, '1240271100439', 280),
(136, '1240271400381', 282),
(137, '1240271700387', 293),
(138, '1240272000398', 309),
(139, '1240272300413', 325),
(140, '1240272600425', 340),
(141, '1240272900431', 356),
(142, '1240273200440', 367),
(143, '1240273500450', 362),
(144, '1240273800382', 360),
(145, '1240274100389', 362),
(146, '1240274400386', 356),
(147, '1240274700390', 352),
(148, '1240275000394', 351),
(149, '1240275300442', 344),
(150, '1240275600427', 336),
(151, '1240275900458', 345),
(152, '1240276200451', 348),
(153, '1240276500439', 351),
(154, '1240276800485', 354),
(155, '1240277100441', 364),
(156, '1240277400387', 360),
(157, '1240277700405', 357),
(158, '1240278000398', 351),
(159, '1240278300416', 358),
(160, '1240278600416', 355),
(161, '1240278900415', 341),
(162, '1240279200434', 343),
(163, '1240279500468', 340),
(164, '1240279800445', 353),
(165, '1240280100607', 355),
(166, '1240280400526', 352),
(167, '1240280700492', 347),
(168, '1240281000410', 339),
(169, '1240281300408', 346),
(170, '1240281600423', 341),
(171, '1240281900464', 620),
(172, '1240282200474', 588),
(173, '1240282500462', 558),
(174, '1240282800468', 524),
(175, '1240283100532', 491),
(176, '1240283400512', 487),
(177, '1240283700516', 475),
(178, '1240284000527', 453),
(179, '1240284300576', 432),
(180, '1240284600495', 390),
(181, '1240284900478', 372),
(182, '1240285200477', 381),
(183, '1240285500551', 365),
(184, '1240285800490', 360),
(185, '1240286100486', 365),
(186, '1240286400482', 356),
(187, '1240286700549', 356),
(188, '1240287000547', 355),
(189, '1240287301570', 360),
(190, '1240287600564', 368),
(191, '1240287900510', 369),
(192, '1240288200468', 370),
(193, '1240288500542', 361),
(194, '1240288800493', 355),
(195, '1240289100545', 347),
(196, '1240289400515', 339),
(197, '1240289700487', 337),
(198, '1240290000540', 314),
(199, '1240290300486', 315),
(200, '1240290600517', 317),
(201, '1240290900521', 313),
(202, '1240291200532', 320),
(203, '1240291500511', 328),
(204, '1240291800482', 328),
(205, '1240292100486', 331),
(206, '1240292400512', 327),
(207, '1240292700493', 325),
(208, '1240293000509', 325),
(209, '1240293300495', 338),
(210, '1240293600505', 334),
(211, '1240293900543', 317),
(212, '1240294200540', 318),
(213, '1240294500534', 314),
(214, '1240294800523', 320),
(215, '1240295100513', 308),
(216, '1240295400488', 305),
(217, '1240295700500', 312),
(218, '1240296000499', 308),
(219, '1240296300494', 315),
(220, '1240296600530', 314),
(221, '1240296900569', 331),
(222, '1240297200518', 326),
(223, '1240297500520', 327),
(224, '1240297800621', 337),
(225, '1240298100573', 336),
(226, '1240298400523', 334),
(227, '1240298700533', 346),
(228, '1240299000492', 337),
(229, '1240299305396', 341),
(230, '1240299605303', 347),
(231, '1240299905283', 347),
(232, '1240300205287', 334),
(233, '1240300505298', 336),
(234, '1240300805322', 325),
(235, '1240301105308', 325),
(236, '1240301405322', 329),
(237, '1240301705309', 342),
(238, '1240302005318', 341),
(239, '1240302305331', 340),
(240, '1240302605323', 342),
(241, '1240302905333', 328),
(242, '1240303205353', 338),
(243, '1240303505346', 348),
(244, '1240303805402', 335),
(245, '1240304105350', 330),
(246, '1240304405390', 330),
(247, '1240304705378', 342),
(248, '1240305005381', 343),
(249, '1240305305402', 336),
(250, '1240305605391', 337),
(251, '1240305905427', 343),
(252, '1240306205405', 337),
(253, '1240306505464', 327),
(254, '1240306805409', 333),
(255, '1240307105428', 344),
(256, '1240307405439', 349),
(257, '1240307705499', 343),
(258, '1240308005490', 339),
(259, '1240308305424', 360),
(260, '1240308605491', 360),
(261, '1240308905458', 362),
(262, '1240309205446', 361),
(263, '1240309505484', 368),
(264, '1240309805456', 372),
(265, '1240310105495', 374),
(266, '1240310405466', 396),
(267, '1240310705499', 410),
(268, '1240311005474', 413),
(269, '1240311305487', 413),
(270, '1240311605603', 418),
(271, '1240311905521', 416),
(272, '1240312205507', 422),
(273, '1240312505508', 421),
(274, '1240312805553', 409),
(275, '1240313105975', 397),
(276, '1240313405919', 386),
(277, '1240313705576', 380),
(278, '1240314005530', 382),
(279, '1240314305559', 392),
(280, '1240314605566', 380),
(281, '1240314905568', 381),
(282, '1240315205542', 371),
(283, '1240315505582', 378),
(284, '1240315805575', 382),
(285, '1240316105579', 394),
(286, '1240316405568', 399),
(287, '1240316705573', 394),
(288, '1240317005574', 390),
(289, '1240317305586', 396),
(290, '1240317605593', 391),
(291, '1240317905594', 376),
(292, '1240318205600', 365),
(293, '1240318505596', 353),
(294, '1240318805612', 335),
(295, '1240319105608', 329),
(296, '1240319405637', 325),
(297, '1240319705611', 315),
(298, '1240320005635', 311),
(299, '1240320305616', 295),
(300, '1240320605628', 289),
(301, '1240320905634', 285),
(302, '1240321205650', 273),
(303, '1240321505634', 266),
(304, '1240321805638', 250),
(305, '1240322105656', 248),
(306, '1240322405646', 233),
(307, '1240322705656', 227),
(308, '1240323005655', 208),
(309, '1240323305659', 204),
(310, '1240323605666', 195),
(311, '1240323905727', 188),
(312, '1240324205677', 183),
(313, '1240324505676', 174),
(314, '1240324805683', 164),
(315, '1240325105683', 156),
(316, '1240325405689', 156),
(317, '1240325705695', 150),
(318, '1240326005696', 139),
(319, '1240326305701', 131),
(320, '1240326605707', 127),
(321, '1240326905710', 117),
(322, '1240327205741', 111),
(323, '1240327505718', 101),
(324, '1240327805722', 101),
(325, '1240328105737', 97),
(326, '1240328405795', 92),
(327, '1240328705737', 82),
(328, '1240329005769', 81),
(329, '1240329305767', 75),
(330, '1240329605752', 69),
(331, '1240329905756', 65),
(332, '1240330205763', 59),
(333, '1240330505772', 53),
(334, '1240330805769', 47),
(335, '1240331105774', 46),
(336, '1240331405779', 46),
(337, '1240331705791', 41),
(338, '1240332005839', 39),
(339, '1240332305792', 36),
(340, '1240332605802', 34),
(341, '1240332905810', 32),
(342, '1240333205807', 28),
(343, '1240333505822', 25),
(344, '1240333805819', 27),
(345, '1240334105820', 23),
(346, '1240334405832', 25),
(347, '1240334705839', 20),
(348, '1240335005846', 20),
(349, '1240335305842', 18),
(350, '1240335605855', 18),
(351, '1240335905853', 21),
(352, '1240336205864', 25),
(353, '1240336505878', 23),
(354, '1240336805876', 24),
(355, '1240337105878', 24),
(356, '1240337405882', 22),
(357, '1240337705881', 23),
(358, '1240338005894', 22),
(359, '1240338305929', 21),
(360, '1240338605937', 20),
(361, '1240338905911', 23),
(362, '1240339205911', 22),
(363, '1240339505917', 20),
(364, '1240339805913', 20),
(365, '1240340105922', 18),
(366, '1240340405922', 17),
(367, '1240340705927', 18),
(368, '1240341005929', 17),
(369, '1240341305934', 18),
(370, '1240341605939', 16),
(371, '1240341905943', 17),
(372, '1240342205950', 19),
(373, '1240342505958', 19),
(374, '1240342805960', 21),
(375, '1240343105965', 20),
(376, '1240343405970', 24),
(377, '1240343705986', 21),
(378, '1240344005978', 22),
(379, '1240344305982', 21),
(380, '1240344605990', 21),
(381, '1240344906002', 22),
(382, '1240345206005', 24),
(383, '1240345506008', 26),
(384, '1240345806010', 25),
(385, '1240346106014', 25),
(386, '1240346406024', 30),
(387, '1240346706031', 32),
(388, '1240347006033', 35),
(389, '1240347306041', 42),
(390, '1240347606042', 45),
(391, '1240347906048', 44),
(392, '1240348206055', 46),
(393, '1240348506105', 51),
(394, '1240348806085', 62),
(395, '1240349106073', 70),
(396, '1240349406078', 70),
(397, '1240349706076', 73),
(398, '1240350006083', 78),
(399, '1240350306089', 83),
(400, '1240350606111', 88),
(401, '1240350906103', 96),
(402, '1240351206106', 100),
(403, '1240351506119', 102),
(404, '1240351806116', 108),
(405, '1240352106127', 118),
(406, '1240352406146', 125),
(407, '1240352706139', 142),
(408, '1240353006205', 149),
(409, '1240353306149', 154),
(410, '1240353606157', 160),
(411, '1240353906184', 169),
(412, '1240354206184', 192),
(413, '1240354506184', 212),
(414, '1240354806178', 218),
(415, '1240355106189', 230),
(416, '1240355406217', 241),
(417, '1240355706200', 243),
(418, '1240356006216', 242),
(419, '1240356306205', 257),
(420, '1240356606229', 267),
(421, '1240356906219', 257),
(422, '1240357206220', 266),
(423, '1240357506239', 258),
(424, '1240357806243', 272),
(425, '1240358106294', 277),
(426, '1240358406237', 277),
(427, '1240358706302', 290),
(428, '1240359006266', 305),
(429, '1240359306267', 296),
(430, '1240359606277', 294),
(431, '1240359906271', 290),
(432, '1240360206315', 286),
(433, '1240360506278', 283),
(434, '1240360806292', 283),
(435, '1240361106290', 288),
(436, '1240361406310', 282),
(437, '1240361706288', 268),
(438, '1240362006323', 289),
(439, '1240362306313', 301),
(440, '1240362606389', 316),
(441, '1240362906330', 331),
(442, '1240363206329', 335),
(443, '1240363506345', 351),
(444, '1240363806387', 343),
(445, '1240364106338', 340),
(446, '1240364406353', 340),
(447, '1240364706348', 344),
(448, '1240365006383', 340),
(449, '1240365306368', 345),
(450, '1240365606372', 345),
(451, '1240365906372', 337),
(452, '1240366206423', 348),
(453, '1240366506378', 350),
(454, '1240366806416', 352),
(455, '1240367106428', 349),
(456, '1240367406448', 346),
(457, '1240367706404', 342),
(458, '1240368006436', 328),
(459, '1240368306421', 326),
(460, '1240368606462', 320),
(461, '1240368906436', 329),
(462, '1240369206427', 331),
(463, '1240369506430', 339),
(464, '1240369806443', 347),
(465, '1240370106443', 348),
(466, '1240370406445', 349),
(467, '1240370706455', 351),
(468, '1240371006464', 346),
(469, '1240371306468', 335),
(470, '1240371606464', 321),
(471, '1240371906476', 315),
(472, '1240372206476', 322),
(473, '1240372506495', 309),
(474, '1240372806491', 306),
(475, '1240373106501', 316),
(476, '1240373406507', 322),
(477, '1240373706537', 318),
(478, '1240374006515', 310),
(479, '1240374306519', 307),
(480, '1240374606582', 300),
(481, '1240374906531', 298),
(482, '1240375206542', 287),
(483, '1240375506540', 280),
(484, '1240375806540', 285),
(485, '1240376106552', 280),
(486, '1240376106553', 280),
(487, '1240376106554', 280),
(488, '1240376106555', 280),
(489, '1240376106560', 270),
(490, '1240376106570', 50),
(491, '1240376106580', 50),
(492, '1240376106590', 15),
(494, '1240376106590', 15),
(495, '1240376106600', 49);

Open in new window

webct-monitor.txt
0
 
LVL 9

Expert Comment

by:richdiesal
ID: 24227688
I think there's something odd going on with how that log is being written.  Look at this section:
(465, '1240370106443', 348),
(466, '1240370406445', 349),
(467, '1240370706455', 351),
(468, '1240371006464', 346),

Generally, these timestamps fall about 300000 points apart.  I'm not sure of the scale of your timestamps here, but they look like UNIX epoch timestamps with three digits appended to the end, which might be thousandths of a second.  So generally, I'm guessing these timestamps fall about every 5 minutes.

But the last timestamps don't.  Look at this section:

(486, '1240376106553', 280),
(487, '1240376106554', 280),
(488, '1240376106555', 280),
(489, '1240376106560', 270),
(490, '1240376106570', 50),
(491, '1240376106580', 50),

It looks like you're getting a timestamp every 5-10 milliseconds here, assuming my conversion is correct.  When I look at the raw logs, that looks accurate - the timestamp for those ("Wed Apr 22 14:55:06 EST 2009") are all identical.  So something odd is going on.

Here's something we can try.  Run this script and post the text file that it outputs (not a copy/paste from a web browser - just save as and then attach).
<?php
require_once('connection/connect.php');
mysql_select_db($database, $connection);
 
$query = "SELECT * FROM node_a_data ORDER BY node_a_data_id DESC LIMIT 1";
$result = mysql_query($query, $connection) or die(mysql_error());
$row = mysql_fetch_assoc($result);
$latest = $row['timestamp'];
 
$log = "webct_monitor.log";
$openLog = fopen($log, 'r') or exit('Unable to open file!');
 
while(!feof($openLog)) {  
	$newline=fgets($openLog);
	if(strstr($newline, 'timestamp=')) {   
		$newline1 = explode('timestamp=', $newline);  
		$substring = explode('time=', $newline1[1]); 
		$timeStamp = str_replace('"','',$substring[0]);
        } elseif(strstr($newline, 'Current user count:')) {
                $substring = explode('Current user count:', $newline);  
                $userCount = $substring[1];  
                if ($timeStamp > $latest) {
                        echo "insert: timeStamp = $timeStamp, latest = $latest, userCount = $userCount\n";
			//$query = "INSERT INTO node_a_data (node_a_data_id,timestamp,user_count) VALUES ('','$timeStamp','$userCount')";
			//$result = mysql_query($query, $connection) or die(mysql_error());
                } else {
			echo "no insert: timeStamp = $timeStamp, latest = $latest, userCount = $userCount\n";
		}
        }  
}  
fclose($openLog);  
?>

Open in new window

0
 

Author Comment

by:peterbrowne
ID: 24230656
The inconsistant timestamp entries at the end of the log is just me copying and pasting additional entries and changing the timestamps to something bigger for testing.

Here is the output attached:
output.txt
0
 
LVL 9

Accepted Solution

by:
richdiesal earned 2000 total points
ID: 24232485
Based on that output, try this.
<?php
require_once('connection/connect.php');
mysql_select_db($database, $connection);
 
$query = "SELECT * FROM node_a_data ORDER BY node_a_data_id DESC LIMIT 1";
$result = mysql_query($query, $connection) or die(mysql_error());
$row = mysql_fetch_assoc($result);
$latest = $row['timestamp'];
 
$log = "webct_monitor.log";
$openLog = fopen($log, 'r') or exit('Unable to open file!');
 
while(!feof($openLog)) {  
	$newline=fgets($openLog);
	if(strstr($newline, 'timestamp=')) {   
		$newline1 = explode('timestamp=', $newline);  
		$substring = explode('time=', $newline1[1]); 
		$timeStamp = trim(str_replace('"','',$substring[0]));
        } elseif(strstr($newline, 'Current user count:')) {
                $substring = explode('Current user count:', $newline);  
                $userCount = $substring[1];  
                if ($timeStamp > $latest) {
                        $query = "INSERT INTO node_a_data (node_a_data_id,timestamp,user_count) VALUES ('','$timeStamp','$userCount')";
                        $result = mysql_query($query, $connection) or die(mysql_error());
                }
        }  
}  
fclose($openLog);  
?>

Open in new window

0
 

Author Comment

by:peterbrowne
ID: 24238114
Beautiful!  I added an extra entry at the end of the log and the script just added that entry (after the first time I ran it when it added all entries).  So it works!  Thank you very much!

Now, what was the change in the code that was the problem?  Was it the trim missing...could you explain how this made it work?

Thanks again!

Peter
0
 
LVL 9

Expert Comment

by:richdiesal
ID: 24238133
The last entry in the output file was this:
insert: timeStamp = 1240376106600 , latest = 1240376106600, userCount = 49

Since that doesn't make sense if you treat both numbers as integers, that means PHP was treating $timeStamp as a string.  So when it checked $timeStamp against $latest, it was comparing strings: "1240376106600 " vs. "1240376106600" and found the first to be later alphabetically because of the empty space at the end.

Trim removes whitespace (blank spaces, new lines, carriage returns, tabs, etc).  So to address the above problem, I added trim to cut 1240376106600+EmptySpace down to just 1240376106600.  Then when it checked $timeStamp and $latest, it checked them as two integers and found them appropriately equal.

You could have accomplished the same thing by using substr to cut the white space off the end, or deleting the space some other way, but trim is probably safer.
0
 

Author Comment

by:peterbrowne
ID: 24238140
Excellent!  Thanks again.

Peter
0

Featured Post

VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article discusses four methods for overlaying images in a container on a web page
This holiday season, we’re giving away the gift of knowledge—tech knowledge, that is. Keep reading to see what hacks, tips, and trends we have wrapped and waiting for you under the tree.
The viewer will learn how to look for a specific file type in a local or remote server directory using PHP.
The viewer will learn how to create and use a small PHP class to apply a watermark to an image. This video shows the viewer the setup for the PHP watermark as well as important coding language. Continue to Part 2 to learn the core code used in creat…
Suggested Courses
Course of the Month17 days, 6 hours left to enroll

864 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question