Some days ago a friend told me, "Ey! Why you didn't write a post talking about how Parsero has been included in the Kali Linux repository?" "Seriously? I forgot it..." So here it is...

As you already know, Kali Linux is one of the most advanced and versatile penetration testing distribution ever made. Kali Linux originally started with earlier version of live Linux distribution named BackTrack. It is a GPL-compliant Linux distribution built by penetration testers for penetration tester. With millions of downloads, it has become the most widely adopted penetration testing framework in existence and is used by the security community all over the world.

That is the reason why I am really proud of announcing that my tool Parsero has been included in the Kali Linux repositories: http://tools.kali.org/information-gathering/parsero

Parsero is a free script written in Python which reads the Robots.txt file of a web server and looks at the Disallow entries. The Disallow entries are the URL path of directories or files hosted on a web server which the administrators don't want to be indexed by crawlers. For example, "Disallow: /portal/login" don't allow to search engines like Google, Bing, Yahoo to index  www.example.com/portal/login  so nobody can locate it by searching on them.

Sometimes these paths typed in the Disallows entries are directly accessible by the users (without using a search engine) just visiting the URL and the Path. Sometimes they are not available to be visited by anybody... Because it is really common that the administrators write a lot of Disallows and some of them are available and some of them are not, you can use Parsero in order to check the HTTP status code of each Disallow entry in order to check automatically if these directories are available or not.

Also, the fact that the administrator write a Robots.txt doesn't mean that the files or directories typed in this file will not be indexed by Bing, Google, Yahoo... For this reason, Parsero is capable of performing searches in Bing to locate content indexed without the web administrator authorization.

Now, you can run Parsero v0.75 directly from this awesome distribution. So, what do you need to use Parsero in Kali Linux?

Installing Parsero in Kali Linux

First of all, you need to execute:

root@kali:~# apt-get update


Then, you can search directly Parsero in the Kali Linux repositories by using the command bellow:

root@kali:~# apt-cache search parsero



Finally run the following command to install it.

root@kali:~# apt-get install parsero


Now you can have fun by checking the directories or files which could have sensitive information and should be "anonymous" to the search engines...


Currently, I'm working on developing the new release which will have another feature. It will be available here: https://github.com/behindthefirewalls/Parsero