TDSのロボットよけ記述

TDS

Openbot,とだけ書いておけばおけです。参考に僕のUA Listのrobotのところを書いておきますね。


ArchitextSpider,
bumblebee@relevare.com,
DIIbot,
DownLoad-http.perl,
Drip,
FAST-WebCrawler,
FastCrawler 3.0.1 (crawler@1klik.dk) - http:,
Googlebot,
httpagent,
HyperRobot,
ia_archiver,
indexpert,
InfoSeek Sidewinder,
Inktomi Search,
InterNet-Html-Searcher,
InternetLinkAgent,
Iron33,
larbin larbin1.1.3@domain.com,
Link2Go,
Mercator,
MuscatFerret,
NEC-MeshExplorer,
NMart_ROBOT(http_srch),
Nutscrape,
Openfind,
Papalink,
Papanda URL_Captor,
PerMan Surfer,
PlantyNet_WebRobot_V1.9 chang@plantynet.com,
Pockey,
polybot 1.0,
RaBot,
Robot,
SpaceBison,
sprocket,
TANSEKI,
TDS_UserInfo_Checker,
TDS_Version_Checker,
tivraSpider,
True_Robot,
W3C_Validator,
WDG_Validator,
WebAuto,
WebBoltac,
WebCrawler,
WebFetch,
Wget,
WhizBang! Lab,
WiseWire-Spider,
WWWC,
WWWD,
WWWOFFLE,
ZyBorg,
Openfind data gatherer,
cosmos,
mogimogi,
SlySearch,
TurnitinBot,
Overture-WebCrawler,
{
	log: no;
	cookie: no;
}


FAST-WebCrawler,
gazz,
InfoNaviRobot,
Lycos_Spider,
Lycos_Spider_,
moget,
Scooter,
Scooter-1.0,
Scooter-3.0.FS,
Scooter-3.0.JB,
Spider,
suke,
DiaGem,
Openbot,
Openfind data gatherer,
minibot,
dloader(NaverRobot),
{
	log: no;
	cookie: no;
	style: no_anchor;
}