• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 420
  • Last Modified:

How to write a scanner program for a compiler?

I have this simple program saved in a text file:
{ Sample program
  in TINY language-
  computes factorial
}
read x; { input an integer }
if 0 < x then { don't compute if x <= 0 }
  fact := 1;
  repeat
    fact := fact * x;
      x := x - 1
      until x = 0;
      write fact { output factorial of x }
end

I need to figure out how to write a program that will read this file, pick out the tokens, and then write the tokens only to another text file.  Please, can anyone give me some insight on how to even start this project!!!
0
morales7_0
Asked:
morales7_0
1 Solution
 
Jaime OlivaresCommented:
Is a long question. You have to learn some about parsing techniques, first.
Have a look to these articles:
String splitting:  http://www.codeproject.com/string/splitstring.asp
Simple math parser: http://www.codeproject.com/string/SParser.asp
Math parser: http://www.codeproject.com/cpp/MathieuMathParser.asp
Advanced C++ parser: http://www.codeproject.com/vcpp/stl/introduction_spirit.asp

0
 
avizitCommented:
for the above simple project you can use lex ,

http://www.cs.rug.nl/~jjan/vb/lextut.pdf

0
 
itsmeandnobodyelseCommented:
Do you mean that?

#include <vector>
#include <string>
#include <iostream>
#include <fstream>

using namespace std;

int main(int nArgs, char* pszArgs[])
{
      vector<string> tokens;
      string token;
      ifstream ifs("input.txt");
      while (!ifs.fail())
      {
           ifs >> token;
           if (ifs.fail() || token.empty())
               continue;

          tokens.push_back(token);
      }

      ifs.close();
      ofstream ofs("output.txt");
      for (int i  = 0; i < tokens.size(); ++i)
           ofs << tokens[i] << endl;
      ofs.close();
       return 0;
}

Regards, Alex
0
VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

 
grg99Commented:
First we need to know if you're supposed to write the scanner from scratch, or whether you are allowed to use any canned tokenizer tools, like lex, yacc, or token objects.

If you have to write it from scratch, here's the basic outline:


   get next input character

repeat
  if it is a left curly brace:  repeat getting next character until EOF or you find a right curly brace.
  if it is a digit:  keep getting input characters until you get a non-digit: return the digits as a kind "number"
  if it is a letter:  keep getting input characters until you get a non-letter: return the letters as a kind "word"
   if it is a space, tab, or end-of-line:  keep getting characters until you get something that isnt a space, tab, or end of line.
   if it is a colon:  get the next character, if it is a equals, return kind = "assignment operator", otherwise return "error"
   if it is anything else: return the character as kind = "operator"
   get next character
 until EOF


0
 
morales7_0Author Commented:
I'm sorry fellows, the program is from scratch and it has to be written in C++.  I should have wrote that in my details of the question.  For now, the program should only be able to recognize the tokens.  The next program I have to do later on is going to be the parse program and I have implement this scan program with it.  
0
 
avizitCommented:
LCC has a implementation of scanner and a parser.

http://www.cs.princeton.edu/software/lcc/

Its in C though , so you have to read andunderstand and then make a C++ version out of it.
The source code is available from the website, but its preferable to get the book too. The book is
quite nice to read bout practical computer implementation vis a vis the theory books like the dragon book.
0
 
grg99Commented:
From scratch, eh?  Then you could use my outline above, easily translated into C or C++ code. Less than a page of code.

0

Featured Post

What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.

Tackle projects and never again get stuck behind a technical roadblock.
Join Now