Link to home
Start Free TrialLog in
Avatar of sh2gr
sh2gr

asked on

robots.txt

Hello experts,

I'm configuring my robots.txt for upload. My questions is:
In what order do the robots follow the index in the txt? should I place the part regarding all robots first or after the part for specified agents?

User-agent: *
Disallow: /folder/

User-agent: Googlebot*
Disallow: /*.js$

or should it be

User-agent: Googlebot*
Disallow: /*.js$

User-agent: *
Disallow: /folder/

As I remember, there is a order how they "read" the file

Cheers
Dimi
ASKER CERTIFIED SOLUTION
Avatar of nltech
nltech

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of sh2gr
sh2gr

ASKER

Thank you nltech,

that helped.

Dimi