Link to home
Start Free TrialLog in
Avatar of sydneyguy
sydneyguyFlag for Australia

asked on

ncrawler and which version of c# does it run i have tried vs005 vs#2008 express vs@2010

the only version that loads is 2010 and throws up a load of errors see below
documentation is as always a bit light so any ideas would be good to get me going
am attempting to run the Net 3.5 version

 Error    1    The type 'System.Lazy<T>' exists in both 'c:\aa new hftfiles\ncrawleer\ncrawler-58912\Net 3.5\NCrawler\bin\Debug\NCrawler.dll' and 'c:\Program Files\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0\mscorlib.dll'    C:\aa new hftfiles\ncrawleer\ncrawler-58912\Net 4.0\NCrawler.HtmlProcessor\Extensions\ExtendedHtmlUtility.cs    19    27    NCrawler.HtmlProcessor
Error    2    The type 'System.Lazy<T>' exists in both 'c:\aa new hftfiles\ncrawleer\ncrawler-58912\Net 3.5\NCrawler\bin\Debug\NCrawler.dll' and 'c:\Program Files\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0\mscorlib.dll'    C:\aa new hftfiles\ncrawleer\ncrawler-58912\Net 4.0\NCrawler.HtmlProcessor\EMailEntityExtractionProcessor.cs    31    27    NCrawler.HtmlProcessor
Error    3    Metadata file 'C:\aa new hftfiles\ncrawleer\ncrawler-58912\Net 3.5\NCrawler.HtmlProcessor\bin\Debug\NCrawler.HtmlProcessor.dll' could not be found    NCrawler.SitemapProcessor
Error    4    Metadata file 'C:\aa new hftfiles\ncrawleer\ncrawler-58912\Net 3.5\NCrawler.HtmlProcessor\bin\Debug\NCrawler.HtmlProcessor.dll' could not be found    NCrawler.Demo
Error    5    Metadata file 'C:\aa new hftfiles\ncrawleer\ncrawler-58912\Net 3.5\NCrawler.HtmlProcessor\bin\Debug\NCrawler.HtmlProcessor.dll' could not be found    NCrawler.Console
Error    6    The type 'System.Lazy<T>' exists in both 'C:\Program Files\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0\mscorlib.dll' and 'C:\aa new hftfiles\ncrawleer\ncrawler-58912\Net 3.5\NCrawler\bin\Debug\NCrawler.dll'    C:\aa new hftfiles\ncrawleer\ncrawler-58912\Net 4.0\NCrawler.HtmlProcessor\RegexSubstitution.cs    12    20    NCrawler.HtmlProcessor
Error    7    The type 'System.Lazy<T>' exists in both 'C:\Program Files\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0\mscorlib.dll' and 'C:\aa new hftfiles\ncrawleer\ncrawler-58912\Net 3.5\NCrawler\bin\Debug\NCrawler.dll'    C:\aa new hftfiles\ncrawleer\ncrawler-58912\Net 4.0\NCrawler.HtmlProcessor\RegexSubstitution.cs    21    18    NCrawler.HtmlProcessor
Error    8    The type 'System.Lazy<T>' exists in both 'C:\Program Files\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0\mscorlib.dll' and 'C:\aa new hftfiles\ncrawleer\ncrawler-58912\Net 3.5\NCrawler\bin\Debug\NCrawler.dll'    C:\aa new hftfiles\ncrawleer\ncrawler-58912\Net 4.0\NCrawler.HtmlProcessor\LinkExtractionProcessor.cs    16    27    NCrawler.HtmlProcessor
Avatar of David Johnson, CD
David Johnson, CD
Flag of Canada image
There are 2 different projects one for .net 3.5 and the other for 4.0.. if you run the build.bat it doesn't use the visual studio at all but msbuild which is included in each version of .net framework.

Tested with vs2010/2012
Avatar of sydneyguy
sydneyguy
Flag of Australia image

ASKER

ok thanks for that have run the 3.5 which seemed to run and build 4.0 did nothing flashed then shut down
there lots of exes now but which do i run to start the program.
is there any useful doc for this program at all
thanks for the help as  well
Avatar of David Johnson, CD
David Johnson, CD
Flag of Canada image
as for .net 4 do you have the full installer for it or just the client install?
http://www.microsoft.com/net/download you need this one
User generated image
the only documentation is on the codeplex site.. minimal documentation there usually.
Avatar of David Johnson, CD
David Johnson, CD
Flag of Canada image
if you run ncrawl.exe from ncrawl.demo
usage: NCrawl url [/threads:number] [/robotrules] [/depth:number] [/connectiontimeout:number] [/timeout:number] [/showtime] [/maximumCrawlCount:number] [/maximumHttpDownloadErrors:number] [/maximumCrawlTime:number] [/userAgent:value]
    [/threads:number]                    Number of concurrent threads to use in crawl
    [/robotrules]                        Adhere to robot rules
    [/depth:number]                      Maximum crawl depth
    [/connectiontimeout:number]          Connection timeout in seconds(default 20)
    [/timeout:number]                    Read timeout in seconds(default 20)
    [/showtime]                          Show download times
    [/maximumCrawlCount:number]          Maximum number of downloads before stopping
    [/maximumHttpDownloadErrors:number]  Maximum number of downloads errors before stopping
    [/maximumCrawlTime:number]           Maximum crawl time in seconds before stopping
    [/userAgent:value]                   User agent, default is NCrawl 2.1
Avatar of sydneyguy
sydneyguy
Flag of Australia image

ASKER

if i run NCrawler\ncrawler-58912\Net 3.5\NCrawler.Demo\bin\Release
it starts then just crashes
but the console box opens text runs then crashes
if i run from NCrawler\ncrawler-58912\Net 3.5\NCrawler.Demo\obj\Release it also crashes but straight away???
Avatar of sydneyguy
sydneyguy
Flag of Australia image

ASKER

which one of these do i need please

For the desktop
Install from the web:

    .NET Framework 4 Client Profile  ›

Download and install:

    .NET Framework 4 Client Profile
    for x86  ›
    .NET Framework 4 Client Profile
    for x86 & x64  ›

For server and development
Install from the web:

    .NET Framework 4 for All CPUs  ›
    Web Platform Installer ›

Download and install:

    .NET Framework 4 for x86  ›
    .NET Framework 4 for x86 & x64  ›
    .NET Framework 4 for x86 & IA-64  ›

Additional resources

    .NET Framework
    System Requirements  ›
    .NET Framework 4 Readme  ›
    What's New in the
    .NET Framework 4  ›
    .NET Framework Deployment
    Guide for Administrators  ›
    .NET Framework Deployment
    Guide for Developers  ›
Avatar of David Johnson, CD
David Johnson, CD
Flag of Canada image
For server and development
Install from the web:

    .NET Framework 4 for All CPUs  ›
    Web Platform Installer ›

Download and install:

    .NET Framework 4 for x86  ›
    .NET Framework 4 for x86 & x64  ›
    .NET Framework 4 for x86 & IA-64  ›

Which x86/x86 and x64 or x86 & IA-64 depends upon which o/s you have installed (32-bit/64bit and what you develop for)  

64bit o/s for x86 and x64
32 bit o/s for x86

up to you whether or not you use the web installer.. or the full installers.
ASKER CERTIFIED SOLUTION
Avatar of David Johnson, CD
David Johnson, CD
Flag of Canada image
Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of sydneyguy
sydneyguy
Flag of Australia image

ASKER

hi ve3ofa
well i now have my system up and going well i can actually update the system now and have v4.0.30319 installed, now when i run the 4.0 bat file
but get a build error please see image
at least we are getting some were now
User generated image
Avatar of sydneyguy
sydneyguy
Flag of Australia image

ASKER

is this a problem with the newer DB
User generated image
Avatar of sydneyguy
sydneyguy
Flag of Australia image

ASKER

thanks ve3ofa for the video, i am guessing one i get the sql server issue rectified then i can get the package going, in your video that you provided you display an environment like visual studio but using msbuild how do you get this environment up and debugging so i can see the code like you have on the screen please
Avatar of sydneyguy
sydneyguy
Flag of Australia image

ASKER

hi ve3ofa i have been playing with this for days how do i get started, there is just no good documentation.
still not sure how to view edit or even run it any help please
Avatar of sydneyguy
sydneyguy
Flag of Australia image

ASKER

thanks for all your help