Understanding Encoding (Beginner’s guide)

From Wikipedia

This article will describe the different type of process involves in encoding of data.

The term encoded data means wrapped data and the process of encoding is used to transform the data into a different format so that it can be easily understood by different type of system. For example ASCII characters are encoded by means of numbers ‘A’ is represented with 65, where as ‘B’ with 66 an so on.

As we known computer does not understand human languages therefore we need to encode the data into binary language which is easily readable by computer systems hence encoding is very important. It utilises such schemes that are widely available so that it can simply be reversed. Encoding means data transformation, not data encryption consequently it does not need a key in decoding.

URL Encoded

The internet only accepts URL’s in ASCII format, URL encoding entails encoding certain parts of the URL character set. This process takes one character and converts it into a character triplet that has a prefix of “%” followed by two digits in hexadecimal format. 

Character Encoded
: %3A
/ %2F
# %23
? %3F
& %24
@ %40
% %25
+ %2B
<space> %20
; %3B
= %3D
$ %26
, %2C
%3C
%3E
^ %5E
` %60
\ %5C
[ %5B
] %5D
{ %7B
} %7D
| %7C
%22

Example :

Original URL: http://www.hackingarticles.in

Encoded URL: http%3A%2F%2Fwww.hackingarticles.in

 Hexadecimal

Hexadecimal or Base 16 is a positional number system which consists of 16 distinct symbols which range from 0 to 9 in numerals and both upper and lowercase alphabets which range from A to F which represent numeric values 10 to 15

Step 1 – is to get the decimal value of an alphabet, this is different for both upper and lower case, eg: A = 65 and a = 97. In order to find the value of any alphabet, we count down to it from ”A” or “a”, the values are in single digit succession, eg: A = 65 B = 66 C = 67 and so on / a = 97 b = 98 c = 99 and so on.

Step 2 – To convert from decimal to hexadecimal, take the decimal value and divide it by 16, the hex value will be written beginning from the quotient all the way up to the remainder. So, the hex value of 97 will be 61.

Eg:

16 97 1
6 6
Source R a j
Decimal Value 82 97 106
Hexadecimal value 52 61 6a

 Base64

Each base64 digit represents exactly 6 bits of data.Is a radix-64 representation of ASCII string, here’s how we get it?

 Step 1 – is to get the decimal value of an alphabet, this is different for both upper and lower case, eg: A = 65 and a = 97. In order to find the value of any alphabet, we count down to it from”A” or “a”, the values are in single digit succession, eg: A = 65 B = 66 C = 67 and so on / a = 97 b = 98 c = 99 and so on.

Step 2 – is to divide the decimal value by 2, where ever there is a reminder it is denoted as “1” and where ever the remainder is “0”, it is denoted as “0”, continue to divide till you reach 0 or 1 and cannot divide any further. The binary value will be the denoted 1’s and 0’s counted from last to first.

Eg:In order to get a 8-bit value we prefix a “0” to the value, eg: 01010010 and this gives us the binary value of “a”.

2 97 1
2 48 0
2 24 0
2 12 0
2 6 0
2 3 1
  1 1

Step 3 – Write the values of all the characters in binary and make pairs of 6 (6-bit), eg: binary value of “Raj” in 8-bit = 010100 100110 000101, binary value of “Raj” in 6-bit = 010100 100110 000101 101010.

Step 4 – Write the 6-bit decimal value of the pairs we make in Step 3 and adding all the values where we have 1’s

32 16 8 4 2 1
0 1 0 1 0 0 20
1 0 0 1 1 0 38
0 0 0 1 0 1 5
1 0 1 0 1 0 42

Step 5 – Use the Base64 table to lookup the values we get in Step 4.

The Base64 index table:

Value Char Value Char Value Char Value Char
0 A 16 Q 32 g 48 w
1 B 17 R 33 h 49 x
2 C 18 S 34 i 50 y
3 D 19 T 35 j 51 z
4 E 20 U 36 k 52 0
5 F 21 V 37 l 53 1
6 G 22 W 38 m 54 2
7 H 23 X 39 n 55 3
8 I 24 Y 40 o 56 4
9 J 25 Z 41 p 57 5
10 K 26 a 42 q 58 6
11 L 27 b 43 r 59 7
12 M 28 c 44 s 60 8
13 N 29 d 45 t 61 9
14 O 30 e 46 u 62 +
15 P 31 f 47 v 63 /

 The Base64 encoded value of Raj is UmFq. Encoded in ASCII, the characters R, a, and j are stored as decimal values 82, 97, and 106, their 8-bit binary values are 01010010, 01100001, and 01101010. These three values are joined together into a 24-bit string, producing 010100100110000101101010. Groups of 6 are converted into individual numbers from left to right. While converting from 8-bit to 6-bit, 0’s are added to fill the last slots, so that a full pair of 6 can be made.

The full conversion of “Raj” to Base64 is shown in Table 1.1 and the individual conversion of “R” and “Ra” of “Raj” are shown in Tables 1.1 and 1.2 to show a breakdown of the process with explanation

Raj                                               82 97 106                             01010010 01100001 01101010


In the Table 1.2, for character “R” of “Raj”, the values in the Bit patternsection are in 8-Bit format and they are being converted into 6-Bit and the decimal value of the 6-Bit pairs are in the Index section.Table 1.1

The same process is repeated in Table 1.3 for characters “R” and “a” of “Raj”.

For each pair of extra 0’s that are added to complete a pair of 6, an “=” is added for each pair, so the ACHII value of “0 0” is “=”.

In table 1.4 to further build on the logic used in table 1.2 and 1.3, “Raaj” is converted to “UmFhag==” in Base64, with the addition of an additional “a”, the complexity of the conversion increases. In the Indexsection we can see an additon of 33, 26 and 32 due to the change in the bit pattern. 

For each pair of extra 0’s that are added to complete a pair of 6, an “=” is added for each pair, so the ACHII value of “0 0” is “=”, as done in table 1.2 and 1.3.

Rot13

This is a letter substitution cypher, it’s conversion process from plain text to cypher test is dicinging the total number of alphabets in half: A to M and N to Z. The first half mirriors the second half and vice versa. So, A = N and N = A.

Eg: Rot13 of Raj = Enw

A B C D E F G H I J K L M
N O P Q R S T U V W X Y Z

 

Understanding HTTP Authentication Basic and Digest

HTTP authentication uses methodologies via which web servers and browsers securily exchanges the credentials like user names and passwords. Http authentication or we can also call it as Digest Authentication follows the predefined methods / standards which use encoding techniques and MD5 cryptographic hashing over HTTP protocol.

In this article we are covering the methodologies/standards used for Http Authentication.

For the sake of understanding we will be using our php scripts that will simply capture user name and passwords and we will generate the Authorization value as per the standards.

For http codes visit here

Basic Access Authentication using Base 64 Encoding

In basic Authentication we will be using base 64 encoding for generating our cryptographic string which contains the information of username and password. Please note we can use any of the encoding techniques like URL, Hexadecimal, or any other we want.

The below example illustrates the concept, we are using Burpsuite for capturing and illustrating the request.

The webpage is asking for input from the client

We are providing “hackingarticles” as User Name and “ignite” as password.

Syntax of basic Authentication

 Value = username:password

Encoded Value =  base64(Value)

Authorization Value = Basic <Encoded Value> 

In basic authentication username and password are combined into a single string using a colon in between.

Value =  hackingarticles:ignite

This string is then encoded using base 64 encoding.

Encoded Value = base64 encoded value of hackingarticles:ignite which is aGFja2luZ2FydGljbGVzOmlnbml0ZQ==

Finally the Authorization Value is obtained by putting the text “Basic” followed by <space> before the encoded value. (We can capture the request using burpsuite to see the result)

The Authorization Value for this example is “Basic aGFja2luZ2FydGljbGVzOmlnbml0ZQ==” . This is the value which is sent to the server.  

Finally the server is decrypting the authorization value and returning the entered credentials

Basic Authentication is less secure way because here we are only using encoding and the authorization value can be decoded, In order to enhance the security we have other standards discussed further.

RFC 2069 Digest Access Authentication

Digest Access Authentication uses the hashing methodologies to generate the cryptographic result. Here the final value is sent as a response value.

RFC 2069 authentication is now outdated now and RFC2617 which is enhanced version of RFC2069 is being used. 

For the sake of understanding the syntax of RFC 2069 is explained below.

Syntax of RFC2069

Hash1=MD5(username:realm:password)

Hash2=MD5(method:digestURI)

response=MD5(Hash1:nonce:Hash2)

Hash1 contains the MD5 hash value of (username:realm:password) where realm is any string

provided by server and username and passwords are the input provided by client.

Hash2 contains the MD5 hash value of (method:digestURI) where method could be get or post depending on the page request and digestURI is the URL of the page where the request is being sent. 

response is the final string which is being sent to the server  and contains the MD5 hash value of (hash1:nounce:hash2) where hash1 and hash2 are generated above and nonce is an arbitrary string that could be used only one time provided by server to the client.

RFC 2617 Digest Access Authentication

RFC 2617 digest authentication also uses MD5 hashing algorithm but the final hash value is generated with some additional parameters

Syntax of RFC2617

Hash1=MD5(username:realm:password)

Hash2=MD5(method:digestURI)

response=MD5(Hash1:nonce:nonceCount:cnonce:qop:Hash2)

Hash1 contains the MD5 hash value of (username:realm:password) where realm is any string

Provided by server and username and passwords are the input provided by client.

Hash2 contains the MD5 hash value of (method:digestURI) where method could be get or post depending on the page request and digestURI is the URL of the page where the request is being sent. 

response is the final string which is being sent to the server  and contains the MD5 hash value of (Hash1:nonce:nonceCount:cnonce:qop:Hash2) where Hash1 and Hash2 are generated above

and for more details on other parameters refer ” https://technet.microsoft.com/en-us/library/cc780170(v=ws.10).aspx”

The actual working of RFC2617 is described below

The webpage is asking for input from the client

We are providing “guest” as User Name and “guest” as password.

Through burpsuite we are capturing the request so that all the parameters could be captured and we can compare the hash values captured with the hash values that we will generate through any other tool (hash calculator in this case).

We have captured the values for the following parameters

realm=”Hacking Articles”, nonce=”58bac26865505″, uri=”/auth/02-2617.php”, opaque=”8d8909139750c6bd277cfe1388314f48″, qop=auth, nc=00000001, cnonce=”72ae56dde9406045″ , response=”ac8e3ecd76d33dd482783b8a8b67d8c1″,

 Hash1 Syntax=MD5(username:realm:password)

hash1 =  md5(guest:Hacking Articles:guest)

The MD5 hash value is calculated as 2c6165332ebd26709360786bafd2cd49

Hash2 Syntax =MD5 (method:digestURI)

Hash2=MD5 (GET:/auth/02-2617.php)

MD5 hash value is calculated as b6a6df472ee01a9dbccba5f5e6271ca8

response Syntax =  MD5(Hash1:nonce:nonceCount:cnonce:qop:Hash2)

response = MD5(2c6165332ebd26709360786bafd2cd49:58bac26865505:00000001:72ae56dde9406045:auth:b6a6df472ee01a9dbccba5f5e6271ca8)

MD5 hash is calculated as  ac8e3ecd76d33dd482783b8a8b67d8c1

Finally the response value obtained through hash calculator is exactly same as that we have captured with burp suit above. 

Finally the server is decrypting the response value and the following is the result

Author: Ankit Gupta, the Author and co-founder of this website, An Ethical Hacker, Telecom Expert, Programmer, India. He Has Found his Deepest Passion To Be Around The World Of Telecom, ISP and Ethical Hacking. Contact Here

Understanding Redirection with Hashing and Crypto Salt (Part 2)

In previous article we have explained the concept of redirection with basic redirection and encoded redirections; in this article we will cover the more secured redirection using hashing and salting techniques.

In this article also we will be covering the redirection using the same php scripts with little modification within the code.

Redirection using Hash Values

On browser type http://localhost/hashing/home.php

Hover on Redirect Link pointing to redirection page (re.php). We can see that the redirection link not only contains the URL as a parameter but also the hash which means that we are not only passing the URL as a parameter but also generating the hash value using MD5/SH1/SHA512 or any of the hashing algorithm and redirection will only work if the combination of url and its hash is correct else not.

(Shown in the figure below). This is a more secure way of redirection.

For the sake of understanding our redirection link is showing the parameters like URL and Hash but in case or real development we can hide them so that attacker won’t be able to judge where the page is being redirected.

When we click on Redirect Link redirection script  on re.php will catch the passed URL and generate its hash value (we are using MD5 hash algorithm) and compare the generated hash value with the hash value we have sent with the request, if both the hash values matches the redirection would work else it will fail.

WE are using hash calculator for generating the MD5 Value of “http://www.hackingarticles.in”.

The MD5 hash value of “http://www.hackingarticles.in” is 8258c1efb05943d059476150cb22df1d

 In the below image we are replacing the original hash value of “http://www.hackingarticles.in”   from its original value which is “8258c1efb05943d059476150cb22df1d” to any different value for example “9258c1efb05943d059476150cb22df1d” (we have replaced only first digit from 8 to 9).

The redirection has failed and script has returned an error message. Finally we are sending the URL along with the generated Hash Value as parameter and result is below

Redirection using Hash Values with salting

On browser type http://localhost/hashing/home.php (page where we have our scripts)

Hover on Redirect Link pointing to redirection page (re.php). Here we can see one more additional parameter salt. As in previous methodology we have worked with Hash values , while working with salting we are introducing  one more parameter salt and generating the hash value of the URL by pre pending or appending the salt value in front of the url or at the end of the url. Salt value could be anything, it could be a combination of characters, digits , alphanumeric , special character or anything we want (In this example we are using the salt value “ignite”). By using salts we are further increasing the security for redirecting the URL.

 For the sake of understanding our redirection link is showing the parameters like URL and Hash and Salt but in case or real development we can hide them so that attacker won’t be able to judge where the page is being redirected.

 When we click on Redirect Link redirection script on re.php will catch the passed URL and generate its hash value (we are using Sha1 hash algorithm) by appending the salt value (ignite) in front of the URL and compare the generated hash value with the hash value we have sent with the request, if both the hash values matches the redirection would work else it will fail.

in Above image we are generating the Sha1 hash value by appending the salt “ignite”  in front of the URL “http://www.hackingarticles.in” , we can use any online/offline convertor in this example we are generating Sha1 hash through http://online-code-generator.com/sha1-hash-with-optional-salt.php

The sha1 hash value of the URL with salt is: 5955e7e3533a0afac6ddfee60a32e2a6731cf626

If the hash value sent is different from the original value our script will return an error. In below Image we are changing the sha1 hash value from 5955e7e3533a0afac6ddfee60a32e2a6731cf626 to 8955e7e3533a0afac6ddfee60a32e2a6731cf626 (we are replacing only first digit from 5 to 8) we will get the following result.

Finally we are sending the URL along with the generated Hash Value as parameter and result is below

Author: Ankit Gupta, the Author and co-founder of this website, An Ethical Hacker, Telecom Expert, Programmer, India. He Has Found his Deepest Passion To Be Around The World Of Telecom, ISP and Ethical Hacking. Contact Here

Understand Hashing in Cryptography (A Practical Approach)

Cryptography is conversion of plain readable text into unreadable form. In cryptography first the data is converted into cipher text (that is encryption) and then the cipher text is converted back into readable form (that is decryption). Cryptography basically works on the concept of encryption and decryption. Encryption and decryption should not be confused with encoding and decoding, in which data is converted from one form to another but is not deliberately altered so as to conceal its content. Encryption is achieved through the algorithms. These algorithms are works with logic, mathematic calculations and its complexities.

Hash Function is most important function in Cryptography. A hash means a 1 to 1 relationship between data. This is a common data type in languages, although sometimes it’s called a dictionary. A hash algorithm is a way to take an input and always have the same output, otherwise known as a 1 to 1 function. An ideal hash function is when this same process always yields a unique output. So you can tell someone, here is a file, and here is its md5 hash. If the file has been corrupted during then the md5 hash will be a different value.

In practice, a hash function will always produce a value of the same size, for instance md5 () is will always return 128bits no matter the size of the input. This makes a 1 to 1 relationship impossible. A cryptographic hash function takes extra precautions in making it difficult to produce 2 different inputs with the same output, this is called a collision. It also makes it difficult to reverse the function. Hash functions are used for password storage because if an attacker where to obtain the password’s hash then it forces the attacker to break the hash before he can use it to login. To break hashes, attackers will take a word list or an English dictionary and find all of the corresponding hash values and then iterate though the list for each password looking for a match.

md5 (), sha0 and sha1 () are all vulnerable to a hash collision attacks and should never be used for anything security related. Instead any member of the sha-2 family, such as sha-256 should be used.

To calculate Hash Value, we will use Hash Calculator. Install Hash Calculator from –> http://www.slavasoft.com/hashcalc/

Hash function plays major role in hacking/forensic world because it helps us to know whether a particular file has changed or not. You can also calculate hash value of your computer and know if anyone has made any kind of changes.

To calculate hash value open Hash Calculator.

Now browse the file of which you want to calculate the hash value. And click on Calculate.

After clicking on calculate it will give too hash values using four different hashing algorithms i.e MD5, SHA1, RIPEMD160, CRC32. You can check other boxes too if you want to use those algorithms to calculate hash value.

This way Hash calculator helps us to know the hash value. Now if there are any changes made in this file, the hash value will change too.

Once I calculated the hash value above i made some changes in the file and calculated the hash value again with the same method and as a result the hash value was changed.

Now, we have two hash values. Let us compare both of these values of MD5. The value of first file is 1110808875326e25dl93e4ee096afaf1 and the value of other file is fb9d53883f302d78c978a583e8a85.

Seeing these two values of MD5 of the same file we can conclude that some changes are made. Because even slightest difference will change the hash value.

But now the main question is how to detect this change because a file can be of 1TB too. Also imagine that you are sending a harddisk full of important documents to someone and there is a huge possibility that someone can bribe the sender and make changes in your documents. So how can you detect these changes?

The answer is very simple –> Compare it! This tool helps us achieve our goal which is to detect the change.

Download Compare it! From —>http://www.grigsoft.com/wincmp3.htm

Open Compare it!

Click file and a drop menu will appear. Select compare files option.

A Dialogue box will open which will ask you to choose the files that you want to compare. Click on Browse button and select your file. And click on Open.

It will show you the changes by highlighting them with green color and the red color will tell the exact change as shown below:

So, in such way you can protect your sensitive data and detect the crime done too.

Author: Yashika Dhir is a passionate Researcher and Technical Writer at Hacking Articles. She is a hacking enthusiast. contact here

Related Posts Plugin for WordPress, Blogger...