Some time ago, I
blogged about the fact that thousands jumped aboard the Newbie Emule Bus
everyday without reading the manual and ensuring their online “safety”.
Safety can be
defined in many forms; there is safety from the various viruses, safety from
spammers, safety from the seedier element online and confidentiality of your
data is a safety measure that should not be overlooked.
Above, all, safety
from prosecution and possibly being jailed is an important consideration, especially if you are convicted of file sharing files that have been placed there by the very same people that are causing you to be convicted..
We start this
article with a Disclaimer -
File sharing has
become a part of Internet culture. Just as speeding is an option of being a
motorist. Just as every road user has the option of speeding/not speeding,
every P2P software user has the option of illegally sharing files or not. Many
elect to partake of the forbidden fruit. It’s a little bit naughty, (sticking it
to “the man” feels good) and for some, distribution of some content is not
available in their country or region for various political or commercial
reasons.
Therefore there is a
validation for many who file share copyrighted content. But not all.
With over 30,000
movies/videos and 25,000 music files in the Public Domain, there is little excuse
to share files illegally and as this article
clearly identifies – is a criminal offense that can in some jurisdictions lead
to jail time.
Each year, an
estimated 20,000 films and a much larger number of music content files are
added to the public domain. (I will blog
at length about how users can identify these public domain files in the near
future. However, to start – you might like to download this-file and absorb
it’s sage wisdom.)
File sharers that
download movies and music for other than fair use are fair game for the growing
interest in file sharing by enforcement agencies globally.
Does this mean that
I am anti file sharing? No. I am very much pro file sharing.
It’s just that
everything that is worthwhile requires hard work. There is no such thing as a
free lunch.
Internet users that
jump onto the network and expect to be able to download the latest movies and
the latest hit tunes for free live in a fantasy world if they do not realise
that their actions are being monitored
with dossiers being collated.
Are there
alternatives to being watched and marked as a person of interest?
Yes, write your
Congressperson, Senator, Member of Parliament. Tell him/her that you don’t want
your children to become automatic criminals for doing what over 57% o the world
is already doing.
Now that we have
advised you of the possible ramifications of your online file sharing activity,
we can start today’s lesson on how you can become less of a “file sharing
dummie”, read on…..
File Sharing is an
activity.
Just like Doctors,
Lawyers and Dentists study for many years to become proficient at their chosen
profession, file sharers need to do a little study too.
P2P based file sharing
programs unfortunately, are not just click and get.
To obtain the best
from your file sharing program of choice, one must invest time.
Intuitive? Yes.
Simple? No.
One must learn about
fake files.
One must learn about
ipfilters or peer guardian.
One must learn about
safe content reference sites instead of relying on the Emule/KAD search engine.
And one must block
denial of service attacks from content industry bots sent out to disrupt all
p2p traffic (not just illegal file sharers) and in fact, illegally, all
internet traffic, even the
traffic of those people that never file share.
We blogged about
Emule being the most downloaded
program in history.
So although every
one claims that Torrent is the most popular file sharing program, until we
receive evidence to the contrary, we will continue to assume that Emule is the
most popular “start-up” learning client out there.
So how many Emule
users are there really ?
Well, our estimates suggest
that on the emule network there may be at any one time an average of about 42
million users. 
(Estimated 30 million 14 million 140 million 230 million 14 million)
However they are segmented
from each other by a form of partitioning or if you like, privacy screens.
Everyone used to
utilize “Servers” to find the files that they wanted.
The Lugundum servers
that used to dominate the indexing “scene” are now so infiltrated with fake
interdiction servers that the majority of users “with some clue” turn off their
server auto update lists.
And manually add
only those servers that have been proven to be trusted.
What is a trusted
server. (http://shortypower.dyndns.org/)
Well, one that doesn’t automatically capture your IP address when uploading or downloading a file that may or not be a copyrighted file and possibly doesn’t introduce to un-trusted users that offer fake and corrupt part files that slow down and detract from your P2P experience.
Last year I uploaded
a 21 MB home movie to my brother in
My mistake was
calling the file Wargames.avi which was an old family joke from the nineties.
My second mistake
was in using Emule because my brother didn’t understand how to set-up or use
FTP.
My third mistake was
that I didn’t realise that the content companies don’t actually check the part
files before Bay/TSP fires off it’s “Safe-harbour” disconnection notices to
ISP’s.
I sent them a bill
for my time and inconvenience but have heard nothing back. Which is what I
would expect from a company
operating illegally in Australia.
So what are the
smart users doing?
Most of the “old
hands” turn off servers and utilize Kademlia.
Kademlia, (which
allows obfuscation) inside Emule is growing at an extra-ordinary rate.
Three years ago,
Kademlia used to show 19k-160k
users 180k-1.5m files.
Now Kademlia shows
1.4,-3.8m users and 155m – 350m files;
And of course, each
user only sees a small microcosm of the Kademlia universe.
When one clicks on
the Kad button – if you are connected, then you will see what appears to be many
duplicates of the following example.
This is a Kademlia
Xor address notation. Users don’t have to understand it to use it. However a
simplified explanation is that short term users are at the 111 end of the “k”
bucket and longer term connected users are at the 000 end of the Kademlia supply
chain.
The Kademlia Supply
Chain (XOR Metric)
And of course, this
is a mere subset. Each “grouping” of kademlia users is separate to each other
“grouping”. Yes there are “gateway” users between the groupings of users, but
Kademlia restricts what is visible with a “reputation” algorythym that is
designed to put you in a random place with random connections that are built up
based on the Reputation Protocol.
|
A reputation protocol operates by nodes granting
service to other nodes based on their reputation within the network, and a
payment protocol operates by having the requesting node make a payment to the
node providing the service. Therefore the more responses you give to queries
and the longer you are connected – the more of the Kademlia universe you can
see and connect too. |
|
The users are
learning.
Installing an
ipfilter.dat file, using Kademlia only, or Kademlia with a trusted server results in a
reasonably safe online p2p experience. Provided of course that one has a
reliable and up to date virus checker. Or if you don’t – don’t download ANY
program files, .wmv files or archive files.
What about the other
280 servers?
Ummmm, fake. Content
Industry spoof servers designed to:
a)
Interdict
your request and provide corrupted data.
b)
Interdict
your request and provide you with a wmv file that requires a licence
c)
Interdict
your request and provide you with a real part file so that the providers colleague/fake
server can obtain the same part back
from you and subsequently send you a take down notice from Bay/TSP.
In other words they are using entrapment to catch file sharers? Yep.
Almost invisible?
That applies only if
you have a low id (i.e.: you operate from a router at home set to something
like 192.168.0.1 and;
If obfuscation is
turned on, that your uploads and downloads look like they are normal http
traffic to your isp.
Does that mean I’m
then safe, if I have a low id and only use safe servers?
Nope. If you use
Kademlia, or servers and are operating from behind a firewall (and you should) then you have a low id and need to have a “buddy” allocated to provide the path through which
your requests are serviced.
Who is your buddy?
Well, basically anyone that has a high “trust” quotient on the network.
Trust is built by
being on for a long period of time (days opposed to hours). Having a high score
(i.e.: uploading/downloading consistently) and a couple of other items that I
don’t wish to reveal in public.
However, the people
that are on the eDonkey (old overnet and now eMule) network the most, the
longest and upload and download the most are in fact the content industry
mercenaries.
So the “Buddy”
system, although necessary, is certainly not a guarantee of anonymity.
So, Koltai, you keep
telling us that it’s dangerous out there……
How, why?
Let us for a moment
examine a logfile of activity (Click Servers then the Verbose Tab) for one
minute from 6:01 pm to 6:02 pm. In that 60 seconds, this logfile shows 37
requests for content from IP numbers that are in the IPFilter.dat file.
All of these IP numbers are from what I call “unfriendlies”.
Each one of which is using my bandwidth to make unfriendly
inquiries about uploading or downloading files.
Some belong to organisations that are collecting statistics,
some to organisations that want to feed me “fake” files and believe it or not,
some to agents of the content companies that actually want to give me real
files.
Why ? A number of reasons.
1. So I can be entraped and later sued.
2. To spread the word - to advertise - to obtain P2P "airtime". (See Payola.)
3. To ensure that File Sharing - their (the content industries) largest money spinner - through litigation and then tax deduction is successful in an era where music only is trailing a poor third fiddle to interactive and alternative media forms.
Wow 37 unfriendlies in a minute. Is that normal?
Yep. If you want to remain in the world of visible file
sharers then that is the price one has to pay.
Because I am collecting statistics I am only curious about
downloads of non-fake files because after all that is what drives the content
industry.
Real content being downloaded and shared by real people which
in turn is promoted to their peers which result in music and artist
appreciation.
Then there are the unfriendlies attempting to gain access to
my program or to discover what version of software I am using:
So how can one stop all these attacks?
My first suggestion is get a good ipfilter.dat file. My
IPFilter.dat is now 28 megabytes.
But then I am mainly only interested in Australian eMule
users. If you live in the
Here’s a good one to get you started http://emulepawcio.sourceforge.net/ipfilter.zip (download it and place the unzipped contents of the file int
your /emule/config directory.
In Part Two – we will talk about “Cleaning up” the Emule
Filebase.
References:
Zhoujun Li, Xiaoming Chen, "Misusing Kademlia Protocol to Perform DDoS Attacks," ispa, pp.80-86, 2008 IEEE International Symposium on Parallel and Distributed Processing with Applications, 2008
http://xlattice.sourceforge.net/components/protocol/kademlia/specs.html
http://en.wikipedia.org/wiki/Kademlia
Implementations
Public clients using the Kademlia algorithm (these networks
are incompatible with one another):
Overnet network: Overnet
(integrated in eDonkey (no longer available) and MLDonkey). With
KadC
a C library for handling its Kademlia is available.
Kad Network: eMule v0.40+, MLDonkey
v2.5-28+. Lphant
v.3.50 beta 2+ and aMule
v2.1.0+.
RevConnect - v0.403+.
BitTorrent Mainline DHT: BitTorrent v4.1.0+, µTorrent
v1.2+, BitSpirit v3.0+, BitComet
v0.59+, KTorrent,
Azureus
3.0+ (via a Plugin), Transmission 1.70+ , BitFlu.pl,
and many libtorrent-based:
They all share a DHT based on an implementation of the Kademlia algorithm, for
trackerless torrents.
Azureus DHT v2.3.0.0+: used for decentralized BitTorrent tracking and various other
features; differing from the BitTorrent Mainline DHT.
Osiris sps (all version): used to
manage distributed and anonymous web portal.
Mojito - a Java Kademlia library written for the LimeWire
project. Mojito is used in LimeWire to provide DHT support for BitTorrent as
well as to augment the Gnutella protocol. See the Class
interface documentation for more information. [3]
Khashmir - Python implementation
of Kademlia. Used in the mainline Bittorrent, with some modifications.
Plan-x - Java
implementation.
SharkyPy - another
python implementation of a Kademlia Distributed Hash Table. LGPL licenced.
Entangled - Python implementation
of Kademlia, also providing a distributed tuple space.
LGPL licenced
RetroShare - Kademlia implementation for
secure Peer-to-Peer messaging and File Sharing
