Tokens and parsing

Andy2002 used Ask the Experts™
Below is the code a program I am writing. It is a simple assembly simulator. I succesfully read memory from a file and store it in an array and set all my registers to 0. I have a file as follows where the first number tells me how many commands are in the file.
load   R1,R8
add   R1,16
loadind   R2,R1
add   R1,8
loadind   R3,R1
add   R3,R2
sub   R1,4
storeind   R3,R1

I have stored the above in the array code[]. Now I want to read each array elemnt and split its contents into tokens - eg load, R1 and R8 would all be tokens. How do I separet them so I can then process what they mean? At present they each line is stored in an array, but i don't know how to sepeare them. Also this is a standard file input, so should be possible for any input file. How do I just read the first line of a file in this case the 8) so i know how many commands to expect?

My whole program is shown below
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

typedef char *STRNG;

STRNG code[20];

int registers[8];

int memory[20];

STRNG get_line(FILE * inf)



int tok = getc(inf);
int count = 0;

s =(char *) malloc(80);

while (tok != '\n' && tok != EOF)

//s[count++] = tok;
s[count++] = (char) tok;

tok = getc(inf);


s[count] = '\0';

return s;


void setRegisters(){
     int i;
     for (i=0; i<8; i++)
          registers[i] = 0;
void getMemory(FILE *inf) {
     STRNG line;
    int i = 0;

     while (strlen(line = get_line(inf)))
   printf ("%s\n", line);
   memory[i++] = atoi(line);

void showMemory(){
     int i;
     for (i=0;i<20;i++){
          printf("%d", memory[i]);}

int main (int argc, char *argv[])
   FILE *data, *assembly;

   if ( argc < 3 )
        printf("Invalid arguments");

   if ( (assembly = fopen(argv[1], "r")) == NULL )
       printf("Cannot open assembly file");

   if ( (data= fopen(argv[2], "r")) == NULL )
       printf("Cannot open data file");


int lines=8;

int count = 0;

for(count =0;count<lines;count++) code[count]= get_line(assembly);

for(count = 0;count<lines;count++) printf("%s\n",code[count]);

return 1;


Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
This should give you the answer you need. However, my browser says that I can only submit a comment, so here it is anyway.

First, to read in a single line from a file:

#include <stdio.h>

FILE * infile;
#define MAX_LINE_LENGTH 255
char buf [MAX_LINE_LENGTH];

   infile = fopen("fname.txt","r");

This will read a line from infile into character array buf for processing. Keep in mind that this function does not replace the new line at the end of the line read with a NULL, so you will need to replace it if needed.

Second, to process the line:

#define DELIM1 'a'
#define DELIM2 'b'
#define DELIM3 'c'
char * first_token;
char * second_token;
char * third_token;

   first_token = strtok(buf,DELIM1);
   second_token = strtok(buf,DELIM2);
   third_token = strtok(buf,DELIM3);

Assuming a line of text such as:


After the calls token_one would point to 'string1', token_two would point to 'string2' and token_three would point to 'string3' (assuming that you did not replace the newline).

If the line of text was: string1astring2astring3a
then you could use the calls:

   first_token = strtok(buf,DELIM1);
   second_token = strtok(buf,NULL);
   third_token = strtok(buf,NULL);

Be aware that strtok() modifies the string (buf) as it goes through it.

to read the first line of the file ..
(i hope u are interested only inthe number .. which says number of lines to follow)

#include <stdio.h>
FILE *fp=fopen("filename.dat","r");
int num;
char buf[32];


printf("%s",buf);//handle each command line



Just use lex or flex on input file and attach it with yacc or bision
lex will separate ur tokens and yacc/bison will parse it.
Why do u want to do all thing by yourself ? Let lex/yacc to do the job for u.
Top Expert 2006

No comment has been added lately and this question is therefore classified abandoned.

If asker wishes to close the question, then refer to

Otherwise, I will leave a recommendation in the Cleanup topic area that this question is:
PAQed with A grade to jstiff

Please leave any comments here within the next seven days. It is assumed that any participant not responding to this request is no longer interested in its final disposition.


EE Cleanup Volunteer

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial