Clan Adverts

Sponsors

CT on Facebook

Latest Product

Battlefield 3 Stats Module
Battlefield 3 Stats Module

User Box

Anonymous
38.107.179.218
Nickname:

Password:

Security Code
Security Code
Type Security Code


PND Downloads Feed

Phpnuke Downloads
How to Delete or Reset yo...
How to Delete or Reset your PHPNuke Admin Account
Lost your admin password ?  Cant login to your admin account ?This sh...
Back Online & Stuff
Back Online & Stuff
Look at us with our nifty website back up and running… Seriousl...
How to edit the admins in...
How to edit the admins in the forums
Here is a way to make someone admin in your forums on your Evo site. I...
Demo Splash Screen
Demo Splash Screen
Ped @ Clan Themes has released a simple splash screen entrance hack. ...
Free Phpnuke Business The...
Free Phpnuke Business Theme 6
This is the 6th Business theme for Phpnuke that Clan Themes has releas...
Tricked Out Slider
Tricked Out Slider
We have seen some great new things come out of Tricked Out News and th...

Custom Work

PHP Nuke Custom Work

www.clanthemes.com :: View topic - Twiceler search bot is a bad spider!!!
Twiceler search bot is a bad spider!!!

1 Replies / 821 Views
Post new topic   Reply to topic  

   www.clanthemes.com Forum Index » Website Issues

View previous topic :: View next topic

5STAR
Reputation: 250 Add RepSubtract Rep
votes: 2
Local time: 2:09 AM
Location: Missouri
usa.gif

0.22 posts per day
Medals: 1 (View more...)
Site Supporter (Amount: 1)
Respected Member
Respected Member
Joined: Nov 06, 2007
Last Visit: 10 Feb 2012
Posts: 346
Points: 36010 
Post Twiceler search bot is a bad spider!!! Posted: Sun Jun 07, 2009 6:29 pm
Shop Purchases:Multi-T3 Multi Gaming Theme · Clan Roster v1.7 Php Nuke Module · Clan Roster 2.0 · Call of Duty 5 World at War Splash · Topsites Manager V1 PhpNuke Module
First off I encourage you to read this. http://www.maximumpc.com/article/news/cuils_twiceler_website_crawlers_causing_plenty_problems_websites

And then take the robots.txt file from the root of your site and add this to the top of the file and then re-upload.
Code:
User-agent: twiceler
Disallow: /


Original post by Phreak at DarkForge Graphics
 

 
View user's profileSend private messageVisit poster's website Reply with quote

clyde4210
Reputation: 395.3 Add RepSubtract Rep
Local time: 3:09 AM

blank.gif

 OMG
OMG

0.11 posts per day
Medals: 0
Joined: Oct 24, 2007
Last Visit: 24 Jul 2009
Posts: 167
Points: 7534 
Post Twiceler search bot is a bad spider!!! Posted: Sun Jun 07, 2009 8:51 pm
Twicler does not follow robots text there for you must use htaccess to deny it. I'm shocked people are just now finding this out as it is 2 years old news.
Code:
RewriteCond %{HTTP_USER_AGENT} Twiceler
RewriteRule ^.*$ http://127.0.0.1 [R,L]


Also contact jim at cuil and ask to be removed and he will do so.
 

 
View user's profileSend private messageVisit poster's website Reply with quote
Post new topic   Reply to topic  
   www.clanthemes.com Forum Index » Website Issues


 
1 Replies / 821 Views
Page 1 of 1
All times are GMT
Display posts from previous:   
 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum
 

Company Information
Home · About Us · Affiliates · Advertising
Link To Us · Newsletter · Contact Us · Site Map
All content © Clan Themes 2006 - 2012
Based on RavenNuke