To own, to be owned, or what else? BT and its proxies.
Two weeks ago my browser started acting weird, where weird means
eating 99% CPU and never loading some pages. At first I ignored it, it
just couldn't load some truecrypt documentation which I could check at work.
But soon it got worse, more pages weren't
loading so I decided to take some time and look into it.
It turns out pages that weren't loading were those either built with frames or
containing iframes. A quick look at the javascript console revealed no errors,
nor the page's source contained anything suspicious. Incidentally that day I wanted to
rip some new CDs but ripperX would fail to look
up info on freedb, which is done via a web request. A coincidence? My attention moved from the
browser to the actual network traffic, and a dump with tcpdump
showed quite a few surprising things. The first attracting my attention was the following header:
Via: 1.0 PSBTTEST:3131 (squid/2.6.STABLE6.2.7-6)Ugh, a proxy! For several reasons I'm an aficionado of whatsmyip.org, which among other things is able to report if you are behind a proxy and give you some information about it by parsing additional headers like X-Forwarded-For. And by that I'm quite confident in saying that this proxy thing didn't exist up to the week before. The reported ip was 217.41.217.22, which a quick whois shows belonging to BT:
inetnum: 217.41.216.0 - 217.41.218.255 netname: BT-MIDBAND descr: BtnMidband country: GBSince BT is able to push updates onto customers' router via CPE CWMP (I'm leaving discussing the security of this solution to another article) I thought they might have tried to improve my internet experience... but I didn't find any differences with the configuration I saved last month. Having ruled out my system and the router, the only possible explanation was a transparent proxy, which as annoying as it can be is still very common and legit.
Being a regular on irc.freenode.net I thought of paying a visit to #squid and have a chat with those nice guys about transparent proxying technologies. And it proved to be useful as I learnt a couple interesting things. The first one is TRACE http method, which can be used in a traceroute fashion, where the Max-Forwards header plays the part of TTL. An example:
spike in space ~ > telnet www.henriknordstrom.net 80 Trying 81.233.163.21... Connected to henriknordstrom.net. Escape character is '^]'. TRACE / HTTP/1.1 Host: www.henriknordstrom.net Max-Forwards: 1 HTTP/1.0 200 OK Date: Sat, 30 Jun 2007 22:59:44 GMT Server: Apache/2.0.40 (Red Hat Linux) Content-Type: message/http Via: 1.0 PSBTTEST:3131 (squid/2.6.STABLE6.2.7-6) Connection: close TRACE / HTTP/1.0 Host: www.henriknordstrom.net Max-Forwards: 0 Via: 1.1 PSBTTEST:3131 (squid/2.6.STABLE6.2.7-6) X-Forwarded-For: 86.130.248.224 Cache-Control: max-age=259200 Connection: keep-alive Connection closed by foreign host.The other thing I learnt is all versions of squid prior to 2.6.12 suffer from a couple serious security vulnerabilities...
But so long nothing new, big telcos running bugged software is unfortunately quite a common practice.
But I was just scratching the surface. Another look at the dump revealed several referrals to dns.sysip.net (212.187.177.142), an ip owned by Level3:
inetnum: 212.187.177.136 - 212.187.177.143 netname: INTEGRATED-COMMUNICATIONS-UK descr: BBBG58761 vlan 396 country: GB admin-c: LTHM tech-c: LTHMA curious thing about this ip is the hostname it resolves to, unknown.Level3.net. Nothing wrong with funny names, we use tons where I work, but still, "unknown" doesnt exactly sound good to me. What sounds even worse is the first page of results you get when googling for dns.sysip.net. Twenty out of twenty links I visited were mentioning it as part of spyware and suggesting various windows software to clean browsers' cache, registry and so on. Now that's a piece of news!
About the spyware
From a network perspective any website I'd connect to, included spikelab.org, would result in an initial GET to the original site immediately followed by a GET to dns.sysip.net, which would download some html/js. Testing against mp3.com:
<html>
<head>
<noscript><meta http-equiv="Refresh" content="0;
URL=http://dns.sysip.net/services/nojscript?orig=http%3A%2F%2Fwww.mp3.com%2F&request-id=1183416188_268148"/></noscript>
<script type="text/javascript"
src="http://dns.sysip.net/services/bind?request-id=1183416188_268148"></script>
<script type="text/javascript">
try {
var domain = location.hostname, cookie_str = "ps-uid=" + UID;
if (!domain.match(/^(\d{1,3}\.){3}(\d{1,3})$/)) {
var h = domain.split("."), l = h.length;
if (l >= 2) {
domain = "." + h[l - 2] + "." + h[l - 1];
if (l >= 3 && (h[l - 2] == "com" || h[l - 2].length < 3))
domain = "." + h[l - 3] + domain;
}
cookie_str += "; domain=" + domain;
}
var d = new Date();
if (UID != "OPTED_OUT") {
d.setTime(d.getTime() + 1000 * 60 * 60 * 24 * 3);
} else {
// set to 4 hours
d.setTime(d.getTime() + 1000 * 60 * 60 * 4);
}
cookie_str += "; path=/; expires=" + d.toGMTString();
document.coo
kie = cookie_str;
} catch(e) { }
window.location = "http://www.mp3.com/";
</script>
</head>
<noscript><iframe
src="http://dns.sysip.net/services/frame?orig=http%3A%2F%2Fwww.mp3.com%2F&request-id=1183416188_268148"
width="1px" height="1px" style="visibility:hidden;">
</iframe></noscript>
</html>
That triggers a http request from the client to dns.sysip.net via the iframe, which results into this:
HTTP/1.0 200 OK Set-Cookie: uid=Khwlm+0o3BGy4AAUXtpNIA||; expires=Thu, 5-Jul-2007 22:43:08 GMT; path=/services/ p3p: CP="NOI DSP LAW CURa DEVa TAIa PSAa PSDa OUR STP BUS UNI COM NAV INT" Date: Mon, 2-Jul-2007 22:43:08 GMT Expires: Thu, 01-Jan-1970 00:00:00 GMT Pragma: no-cache Cache-Control: no-cache Content-Type: application/x-javascript Connection: close var UID="Khwlm+0o3BGy4AAUXtpNIA||";My best guess is they are logging the request plus setting a variable and cookie for tracking purposes. Calling that url manually and fiddling with the request-id parameter results in a similar value in output with the only first 8 chars changing, which suggests some kind of weak encryption schema where the second part is based on the client ip or similar.
On a website with no ads like mine that is the end of it. But on something riddled with ads like mp3.com the story goes on. Further down the dump we can observe:
206.16.21.32:www AP > space.home:57228 (1420)
HTTP/1.1 200 OK
Connection: close
Transfer-Encoding: chunked
p3p: CP="NOI DSP LAW CURa DEVa TAIa PSAa PSDa OUR STP BUS UNI COM NAV INT"
Date: Mon, 2-Jul-2007 22:43:23 GMT
Expires: Thu, 01-Jan-1970 00:00:00 GMT
Pragma: no-cache
Cache-Control: no-cache
Content-Type: text/html
Set-Cookie: ps-origreferer=http://www.mp3.com/; path=/CNT/iview/gmsptcbr1380000276cnt/direct;wi.300;hi.250/01/2007.07.02.18.43.09
Set-Cookie: ps-request-id=1183416203_204989; path=/CNT/iview/gmsptcbr1380000276cnt/direct;wi.300;hi.250/01/2007.07.02.18.43.09
A1F
<html>
<head>
<noscript><meta http-equiv="Refresh" content="0;
URL=http://dns.sysip.net/services/nojscript?orig=http%3A%2F%2Fview.atdmt.com%2FCNT%2Fiview%2Fgmsptcbr1380000276cnt%2Fdirect%3Bwi.300%3Bhi.250%2F01%2F2007.07.02.18.43.09%3Fclick%3Dhttp%3A%252F%252Fadlog%252Ecom%252Ecom%252Fadlog%252Fe%252Fr%253D7166%2526s%253D740157%2526o%253D1%3A%2526h%253Dpi%2526p%253D%2526b%253D4%2526l%253DEn_US%2526site%253D6%2526pt%253D%2526nd%253D%2526pid%253D%2526cid%253D%2526pp%253D%2526e%253D%2526rqid%253D00c17-gne-ad5468237F130864CA%2526t%253D2007.07.02.18.43.09%2526event%253D58%252F&request-id=1183416203_204989"/></noscript>
var domain = location.hostname, cookie_str = "ps-uid=" + UID;
if (!dom
206.16.21.32:www A > space.home:57228 (1420)
ain.match(/^(\d{1,3}\.){3}(\d{1,3})$/)) {
var h = domain.split("."), l = h.length;
if (l >= 2) {
domain = "." + h[l - 2] + "." + h[l - 1];
if (l >= 3 && (h[l - 2] == "com" || h[l - 2].length < 3))
domain = "." + h[l - 3] + domain;
}
cookie_str += "; domain=" + domain;
}
var d = new Date();
if (UID != "OPTED_OUT") {
d.setTime(d.getTime() + 1000 * 60 * 60 * 24 * 3);
} else {
// set to 4 hours
d.setTime(d.getTime() + 1000 * 60 * 60 * 4);
}
cookie_str += "; path=/; expires=" + d.toGMTString();
document.cookie = cookie_str;
} catch(e) { }
window.location =
"http://view.atdmt.com/CNT/iview/gmsptcbr1380000276cnt/direct;wi.300;hi.250/01/2007.07.02.18.43.09?click=http:%2F%2Fadlog%2Ecom%2Ecom%2Fadlog%2Fe%2Fr%3D7166%26s%3D740157%26o%3D1:%26h%3Dpi%26p%3D%26b%3D4%26l%3DEn_US%26site%3D6%26pt%3D%26nd%3D%26pid%3D%26cid%3D%26pp%3D%26e%3D%26rqid%3D00c17-gne-ad5468237F130864CA%26t%3D2007.07.02.18.43.09%26event%3D58%2F";
</script>
</head>
<noscript><iframe src="http://dns.sysip.net/services/frame?orig=http%3A%2F%2Fview.atdmt.com%2FCNT%2Fiview%2Fgmsptcbr1380000276cnt%2Fdirect%3Bwi.300%3Bhi.250%2F01%2F2007.07.02.18.43.09%3Fclick%3Dhttp%3A%252F%252Fadlog%252Ecom%252Ecom%252Fadlog%252Fe%252Fr%253D7166%2526s%253D740157%2526o%253D1%3A%2526h%253Dpi%2526p%253D%2526b%253D4%2526l%2
space.home:57228 A > 206.16.21.32:www (0)
206.16.21.32:www APF > space.home:57228 (308)
14 FIN-WAIT-1 space.home:57228 > 206.16.21.32:www
53DEn_US%2526site%253D6%2526pt%253D%2526nd%253D%2526pid%253D%2526cid%253D%2526pp%253D%2526e%253D%2526rqid%253D00c17-gne-ad5468237F130864CA%2526t%253D2007.07.02.18.43.09%2526event%253D58%252F&request-id=1183416203_204989" width="1px" height="1px" style="visibility:hidden;"></iframe></noscript>
</html>
0
I'm not entirely sure what's happening here, the same technique is used, hidden iframe calling home with one of the ads link as parameter. It seems to do this for all the ads on the page, maybe in some sort of referral-based revenue schema. Here the triggered request:
Host: dns.sysip.net User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.3) Gecko/20061201 Firefox/2.0.0.3 (Ubuntu-feisty) Accept: */* Accept-Language: en-gb,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive Referer: http://view.atdmt.com/CNT/iview/gmsptcbr1380000276cnt/direct;wi.300;hi.250/01/2007.07.02.18.43.09?click=http:%2F%2Fadlog%2Ecom%2Ecom%2Fadlog%2Fe%2Fr%3D7166%26s%3D740157%26o%3D1:%26h%3Dpi%26p%3D%26b%3D4%26l%3DEn_US%26site%3D6%26pt%3D%26nd%3D%26pid%3D%26cid%3D%26pp%3D%26e%3D%26rqid%3D00c17-gne-ad5468237F130864CA%26t%3D2007.07.02.18.43.09%26event%3D58%2F Cookie: uid=Khwlm+0o3BGy4AAUXtpNIA||As you can see the cookie set during the first request is now used.
The other interesting bit is the ip involved in the request, 206.16.21.32 :
CERFnet CERFNET-BLK-206 (NET-206-16-0-0-1)
206.16.0.0 - 206.19.255.255
CERFnet customer - University of La Verne CERF-ULAVERNE (NET-206-16-20-0-1)
206.16.20.0 - 206.16.23.255
An university... how is an university involved with mp3.com and a spyware
domain? The most likely answer to me is a cracked account used to bounce from,
or something along those lines.Another thing worth mentioning, that I believe confirms the role of the proxy, is what happens when you fetch the same page with curl. curl mp3.com | grep sysip gives no results, but add -A "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)" and you're served. I think that here BT is filtering for user-agent in order to save bandwidth and CPU by avoiding serving requests made by robots, which in turn means no spyware.
Two hypothesis
I hate uncertainty but with the data I can collect all I can do is formulating two hypothesis:1) BT is pushing spyware onto its customers
2) BT's proxy has been cracked and used to infect customers
There's actually a third hypothesis based on what's been going on during the last 3 weeks with some ISPs injecting ads into pages by rewriting content on the fly. This doesnt seem the case of BT but they could have adopted some similar but less obvious technique.
An update
At the beginning of the article I mentioned that it all started two weeks ago. Unfortunately I've been away and too busy to produce this report sooner, and at the time of speaking the proxy is gone and there are no longer traces of sysip.net. Based on my dumps it seems the proxy was gone two days ago. Additionally something changed in the routes. Before a tcptraceroute would produce something along the lines of:... ... 5 217.41.171.74 55.955 ms 48.530 ms 43.221 ms 6 217.41.217.114 41.263 ms 41.745 ms 42.060 ms 7 www.spikelab.org (217.147.94.42) [open] 48.709 ms 50.791 ms 61.070 msnow the same command gives me:
... ... 4 192.168.50.253 34.117 ms 38.298 ms 41.161 ms 5 217.41.171.74 55.955 ms 48.530 ms 43.221 ms 6 217.41.171.42 43.639 ms 46.307 ms 59.785 ms ... ... 13 rbii-rt2-e1.blackcat.net.uk (217.147.87.204) 40.021 ms 53.248 ms 48.008 ms 14 spikelab.org (217.147.94.42) [open] 56.927 ms 58.255 ms 46.341 msIt's interesting to also notice the internal ip hop. Such node started showing up two days ago when the problem was "fixed", and during that day it would come up as last node, which I didn't really manage to explain, but which seems to be causing quite a few RST.