Penetration Testing

Comprehensive Guide on Gobuster Tool

Hello Friend!! Today we are going demonstrate URLs and DNS brute force attack for extracting Directories and files from inside URLs and sub-domains from DNS by using “Gobuster-tool”.

Table of Content

  • Introduction & Installation
  • Using Wordlist for Directory Brute-Force
  • Obtaining Full Path for a directory or file
  • Hide Status Code
  • Verbose Mode
  • Identify Content-Length
  • Disable Banner
  • User-Agent Mode
  • Obtain Result with Specify Status Code
  • Timeout
  • Appending Forward slash
  • Saving Output Result inside Text File
  • Enumerating Directory with Specific Extension List
  • Follow Redirect
  • HTTP AUTHORIZATION (-u username: password)
  • DNS Mode
  • Set Threads Number
  • Obtain Sub domain IPs
  • Force Processing Brute Force
  • Hide Process of Extracting
  • Extracting CNAME Records

Introduction & Installation

Gobuster is a tool used to brute-force on URLs (directories and files) in websites and DNS subdomains. Gobuster can be downloaded through the apt- repository and thus execute the following command for installing it.

apt-get install gobuster

When it will get installed, you can interact with it and can perceive all available option with the help of the following command.

gobuster -h

Common Parameters

  • -fw – force processing of a domain with wildcard results.
  • -np – hide the progress output.
  • -m <mode> – which mode to use, either dir or dns (default: dir).
  • -q – disables banner/underline output.
  • -t <threads> – number of threads to run (default: 10).
  • -u <url/domain> – full URL (including scheme), or base domain name.
  • -v – verbose output (show all results).
  • -w <wordlist> – path to the wordlist used for brute forcing (use – for stdin).

Dir mode Parameter

  • -a <user agent string> – specify a user agent string to send in the request header.
  • -c  – use this to specify any cookies that you might need (simulating auth).
  • -e – specify the extended mode that renders the full URL.
  • -f – append / for directory brute forces.
  • -k – Skip verification of SSL certificates.
  • -l – show the length of the response.
  • -n – “no status” mode, disables the output of the result’s status code.
  • -o <file> – specify a file name to write the output to.
  • -p <proxy url> – specify a proxy to use for all requests (scheme much match the URL scheme).
  • -r – follow redirects.
  • -s <status codes> – comma-separated set of the list of status codes to be deemed a “positive” (default: 200,204,301,302,307).
  • -x <extensions> – list of extensions to check for, if any.
  • -P  – HTTP Authorization password (Basic Auth only, prompted if missing).
  • -U  – HTTP Authorization username (Basic Auth only).
  • -to  – HTTP timeout. Examples: 10s, 100ms, 1m (default: 10s).

DNS mode Parameters

  • -cn – show CNAME records (cannot be used with ‘-i’ option).
  • -i – show all IP addresses for the result.

Using Wordlist for Directory Brute-Force

You can use -w option for using a particular wordlist, for example, common.txt or medium.txt to launch a brute-force attack for extracting web directories or files from inside the target URL.

gobuster -u http://192.168.1.108/dvwa -w /usr/share/wordlists/dirb/common.txt

The above command will dump the all possible files and directories with the help of common.txt wordlist.

Obtaining Full Path for a directory or file

Using -e option provides a more significant result, as it Prints complete URL when extracting any file or directories.

gobuster -e -u http://192.168.1.108/dvwa -w /usr/share/wordlists/dirb/common.txt

You can compare the following output result from the previous result.

Hide Status Code

Using -n Option “no status” mode, it prints the output of the results without displaying the status code.

gobuster -u http://192.168.1.108/dvwa -w /usr/share/wordlists/dirb/common.txt -n

The above command will dump all possible files and directory without displaying their status code.

                          

Verbose Mode

Using -v option – it enables the verbose parameter and makes brute-force attack vigorously on each file or directory.

gobuster -u http://192.168.1.108/dvwa -w /usr/share/wordlists/dirb/common.txt -v

As you can observe from the following option that, this time it has to dump the result including status 404 for missing directories or files.

Identify Content-Length

Using -l option enables content-length parameter which displays the size of the response. A Content-Length header is a number denoting and the exact byte length of the HTTP body for extracted file or directory.

gobuster -u http://192.168.1.108/dvwa -w /usr/share/wordlists/dirb/common.txt -l

Disable Banner

Gobuster always adds the banner to specify the brief introduction of applied options while launching a brute force attack. By using -q option we can disable the banner to hide additional information.

gobuster -u http://192.168.1.108/dvwa -w /usr/share/wordlists/dirb/common.txt -q

From the given below image, you can perceive the difference between the last output results and in the current result.

User-Agent Mode

Using -an option enables User-Agent mode to specify a user agent string to send in the request header for extracting directories and files from inside the target URL.

gobuster -u http://192.168.1.108/dvwa -w /usr/share/wordlists/dirb/common.txt -a Mozilla/5.0 -fw

Obtain Result with Specify Status Code

Using -s Option enables the status code for specific value such as 302, 200, 403, and 404 and so on to obtain certain request pages.

gobuster -u http://192.168.1.108/dvwa -w /usr/share/wordlists/dirb/common.txt -s 302
gobuster -u http://192.168.1.108/dvwa -w /usr/share/wordlists/dirb/common.txt -s 200

From the given below image, you can take reference for the output result obtained for the above commands.

Timeout

Using -to option enables the timeout parameter for HTTP request and 10 second is the Default time limit for the HTTP request. 

gobuster -u http://192.168.1.108/dvwa -w /usr/share/wordlists/dirb/common.txt -to 10s

Appending Forward slash

Using -f option, appending the forward slash while making brute-force attack on the target URL.

gobuster -u http://192.168.1.108/dvwa -w /usr/share/wordlists/dirb/common.txt -f

 

Saving Output Result inside Text File

Using -o option enables saving output result parameter in a text file which can be useful in the future.

gobuster -u http://192.168.1.108/dvwa -w /usr/share/wordlists/dirb/common.txt -o result.txt

We can ensure the result.txt file with the help of cat command

cat result.txt

 

Enumerating Directory with Specific Extension List

There are a lot of situations where we need to extract the directories of a specific extension over the target server, and then we can use the -X parameter of this scan. This parameter accepts the file extension name and then searches the given extension files over the target server or machine.

gobuster -u http://192.168.1.108/dvwa -w /usr/share/wordlists/dirb/common.txt -x .php

 

Follow Redirect

Using -r options enables redirect parameter which redirects HTTP request to another and modifies the Status code for a directory or file.

gobuster -u http://192.168.1.108/dvwa -w /usr/share/wordlists/dirb/common.txt -q
gobuster -u http://192.168.1.108/dvwa -r -w /usr/share/wordlists/dirb/common.txt -q

You can compare the output result of the default scan with redirect output result.

HTTP AUTHORIZATION (-u username: password)

HTTP Authentication/Authentication mechanisms are all based on the use of 401-status code and WWW-Authenticate response header. The most widely used HTTP authentication mechanisms are Basic. The client sends the user name and password as un-encrypted base64 encoded text.

So, in order to bypass this kind of authentication with the help of Gobuster we have used the command below:

gobuster -u http://testphp.vulnweb.com/login.php -w /usr/share/wordlists/dirb/common.txt -U test -P test

As a result, it is shown Status –code 200 for the test: test and authorized credential on target URL.

DNS Mode

Using -m option is enabled DNS mode which is effective for public network IP and extracts the sub-domains.

gobuster -m dns -u google.com -w /usr/share/wordlists/dirb/common.txt

As you can observe the output result from the given below result.

Set Threads Number

Using -t option, it enables the number of thread parameter to be applied while brute-forcing sub-domains name or directories.

gobuster -m dns -u google.com -t 100 -w /usr/share/wordlists/dirb/common.txt

Obtain sub-domains IPs

Using -i option enables the IP parameter which should be showing IPs of extracted sub-domains.

gobuster -m dns -u google.com -t 100 -w /usr/share/wordlists/dirb/common.txt -i

From the given below result, you can observe that it showing IPv4 of Ipv6 for each extracted sub-domains.

Force Processing Brute Force

It stops extracting the sub-domains name if meet any Wildcard DNS which is a non-existing domain, therefore uses -fw option to enable force processing parameter to continue the attack even if there is any Wildcard Domain.

gobuster -m dns -u google.com -t 100 -w /usr/share/wordlists/dirb/common.txt -fw

Hide Process of Extracting

Using -np option hides the process of extracting sub-domains name while making brute force attack.

gobuster -m dns -u google.com -t 100 -w /usr/share/wordlists/dirb/common.txt -fw -np

Extracting CNAME Records

Using –cn option enables CNAME Records parameter of the extracted sub-domains and show their CNAME records.

gobuster -m dns -u google.com -t 100 -w /usr/share/wordlists/dirb/common.txt -cn

You can observe the output for above-executed command in the given below result.

Proxy URL

Using –p option enables proxy URL to be used for all requests, by default it works on port 1080. As you can observe, on exploring target network IP in the web browser it put up “Access forbidden error” which means this web page is running behind some proxy.

To ensure this prediction, we run the gobuster command twice, firstly on port 80 which is by default and further on port 3129 along with –p option which enables proxy parameter.

gobuster -u http://192.168.1.108/ -w /usr/share/wordlists/dirb/common.txt
gobuster -u http://192.168.1.108/ -w /usr/share/wordlists/dirb/common.txt –p 192.168.1.108:3129

From the given below image, you can take reference for the output result obtained for above commands, here we haven’t obtained any directory or file on executing the first command where else in the second command executed successfully.

Author: Shubham Pandey is a Technical Writer, Researcher and Penetration tester contact here