|
|
| The spiders that visit your site consumes much of your bandwidth?
If you want to control the robots or spiders that visit your site, robots-txt generator is the solution. |
|
| Updated |
Apr 18, 2009 10:24:16 | | Size | 51 kb | | Licence | Freeware | | Status | New Release | | Languages | English | | Tags | robots txt, robots txt generator, web spider, web bot, bad bot | | OS | Windows ME, Windows NT, Windows 2000, Windows XP, Windows 2003, Windows Vista | | Homepage | robots-txt.info | | Author | ricardo farias |
|
|
 click for full size
|
| Other author's software: | - Mp3Gain PRO 1.04 — Mp3Gain PRO (formerly SuperMp3Normalizer) is a software designed specifically to adjust and normalize the volume of mp3 files.
Mp3Gain PRO is an easy to use software. Normally it is sufficient to click a button to get an optimized volume level
| | Show all author's software |
Description:
robots-txt generator
The spiders that visit your site consumes much of your bandwidth?
If you want to control the robots or spiders that visit your site, robots-txt generator is the solution.
It is very common that the webmaster of each site spends a lot of money paying for bandwidth and it is unfair that a significant portion of that bandwidth is consumed spiders.
Robots txt generator, you can stop or control the bots.
Some robots or spiders saturate your server making countless queries per minute?
With Robots txt generator, you can limit how many requests can make any robot or spider per hour or per minute.
In addition, the program has a database of more than 300 bots, allowing you to search on the database the source and owner of any bot you have found in your logs.
You can even specify different restrictions for different bots. Per example, you can allow google bot, but disallow google-image-bot. You can specify a different restriction level for each one of the 300 bots in the database or can specify some restrition to the "general" bots, plus some specific for some specifi bots. Like giving a ban to some specific bot.
Best of all, is that the software is free! | Short tags:
robots txt, robots txt generator, web spider, web bot, bad bot | System Requirements:
none | |
|  Free Download Robots-txt-generator 1.01 from robots-txt.info |
| Similar software shotlights: | - Oven Fresh Robots.txt Maker 3.6 — Make a Robots.txt File: protect private files from search engine indexing. Do not let visitors become discouraged by clicking on a search engine link and not being able to reach your site. Unlimited…
- Robots.txt Maker 3.8 — Make a Robots.txt File: protect private files from search engine indexing. Do not let visitors become discouraged by clicking on a search engine link and not being able to reach your site. Unlimited…
| | Find all software similar on Robots-txt-generator 1.01 |
| Similar smart reviews: | - Bot Boy: Friends Gotten Together — Free Friend Finder. «A friend in court is better than a penny in purse»,-this is equally true in the real life and on the Internet. Having many friends in your online community is beneficial. The challenge is making friends or, in the online terms, adding friends to your network. With Bot Boy, a completely free friend adder application for MySpace, the…
- YouTube Robot, You Sleep, I Fetch — If you never let your mobile out of your hands and starve for fresh videos from YouTube and similar sites to watch while you’re on the go — what else could be more pleasing than having the show always ready to go on? With YouTube Robot you can forget about recharging your player with fresh movies and just take care of your business, while the software automatically checks your…
- EXECryptor 2.2.0: No Pasaran, Bad Greedy Pirates! — EXECryptor is a powerful, time tested and uncrackable tool for protecting an application against reverse engineering, analysis and modifications. If you are an independent software vendor then you know about software pirates not by hearsay. The words «warez», «crack/patch» and «serial numbers generator» are your nightmares. You are afraid of calculating your losses. Some…
- CYAN Secure Web – a fresh look at web security and content protection — Today’s standards of web security require much more than they did a decade ago. Basic access blocking is not as efficient as it used to be and content has to be reliably protected as well. If you or your company is seeking a rock-solid and versatile web security solution, take a closer look at CYAN Secure Web! CYAN Secure Web is a powerful…
- 3S Web Tools v2.5. Web Developer's Joy. — The website development is a strange thing. Unlike other programming and near-programming activities, webdev requires much more of good taste, common sense, and wish to «do well» rather than strong programming skills. And even when some (or more) of programming IS required, such tasks are rather routine and only require time and accuracy, not «fantasia». So, whenever somebody…
- 3D Graph Generator 2.4, Visualize Smart! — The rapid development of modern computer-graphics hardware and software is a merit of computer games and other entertaining things. However, all those «entertainment-oriented» features can be effectively used for «serious» data visualization and analysis, completely changing the level of understanding of huge data arrays. A good illustration for the thesis is 3D Graph Generator…
- Extreme Thumbnail Generator 1.9.3, The Images, The Thumbnails and The Gallery — The task is common, so, I’ll be short in description. You have a set of images (presumably photos) and want to publish them in the web. You don’t want them to be published like, «John, you can download the archive with photos from my FTP», you would rather have a gallery, with thumbnails for each image on several pages and personal pages for each…
- Audio Multi-Channel Generator - a powerful tool for audio professionals and engineers — When it comes to calibrating complex acoustic equipment, not only do you have to be a real professional, but there should also be tools that will enable you to analyze the current parameters, compare them with reference values and make the necessary adjustments. As a rule, this process requires various specialized measurement tools that may cost you a fortune. Besides, using a bunch…
- Advanced Data Generator - quick and easy generation of test data for your database — Whenever you develop a database-driven application of any type, database modeling and creation are an integral part of the process. However, testing cannot be properly carried out without realistic database data — you don’t see how objects, text and other database elements are rendered on the page, you can’t see a full picture of your e-commerce store full of products and…
| | Find all smart reviews similar on Robots-txt-generator 1.01 |
|
