Summer 2017 Internship to Create a Bridge Bandwidth Scanner
by isis | June 15, 2017
This is a mentored internship position to produce a bridge bandwidth scanner for The Tor Project.
The Tor Network has what are called "Bandwidth Authorities": volunteer-run machines which build circuits through permutations of all the relays in the network to connect back to themselves and request files of different sizes, in order to statistically determine the likely maximum bandwidth capacity of each relay. For the relay bandwidth scanners, the circuit used for testing look like this:
BW → A → B → BW
Where A is the relay being measured, B is a relay believed to possess equal or greater bandwidth than A (otherwise the circuit would bottleneck at B), and BW is the Bandwidth Authority doing the measurement.
The intern is responsible for designing and implementing a similar system for measuring the bandwidth of Tor bridge relays.
Design Constraints
The bridge bandwidth scanner produced should meet the following design/implementation constraints:
Must be implemented in one of the following memory-safe languages:
Python;
Rust;
Another $LANGUAGE, but you'll have a lot of convincing to do.
Must run as a daemon;
Must produce a measured bandwidths file identical (or nearly identical) insyntax to the measured bandwidth files currently produced by relay bandwidthauthorities (sample format).
Please be aware if you choose Rust, that The Tor Project does not yet currently have nearly as many tools and libraries written in Rust. For example, you'll need to implement (at least a fraction of) bridge descriptor parsing (whereas in Python you'll be able to outsource this to Stem) and circuit construction through Tor's ControlPort (also outsourceable in Python to txtorcon). If you choose Rust, I will gladly help you implement these functionalities in separate crates (this will make it easier for us to expand upon them more, later on).
Please also be aware that, while there is better library support in Python, using txtorcon will require knowledge of Twisted, an asynchronous framework known for being… well… twisted.
Other constraints on the project are:
During the course of the work, the intern should attend weekly tor dev meetings(on irc.oftc.net in #tor-dev on Mondays at 17:00 UTC), or otherwise sendbrief weekly status reports should be sent to tor-project@lists.torproject.organd the mentor(s).
The position is remote and may take place in any location of the intern's choice. (Optionally, you're welcome to arrange with your mentor(s) to work in person, but we cannot allocate funds towards travel expenses at this time.)
The length of the internship project is negotiable (between 1 and 3 months), and the (non-negotiable, sorry) compensation is $3000 USD.
Applications must be received by midnight UTC on Monday 26 June 2017.
It is not necessary to be (or have been) a student to apply.
Prerequisite Skills/Knowledge
Reasonable ability to communicate w.r.t. technical matters in English, German, or French (in that order of preference);
Python, Rust, or $LANGUAGE;
Basic knowledge of how a circuit is constructed through the Tor Network;
Basic knowledge of Tor bridges and anti-censorship infrastructure.
Applicants with the following demonstrable skills/knowledge will be prioritised:
Public code samples in the language of choice;
Contributions of (integration) tests to an open source project (again,preferably in the language of choice);
Please apply by sending an email whose subject contains the phrase "Bridge Bandwidth Scanner Internship" and includes the following information toisis@torproject.org:
A brief description of yourself and/or a résumé;
Links to, or attachments of, sample code you've authored;
If you are unable to provide code, please, in the $LANGUAGE you are choosing to do the project in, write a SOCKS5 (RFC 1928) proxy which (assuming there is no encryption, non-trivial encodings, or compression on the underlying protocol being transported, e.g. the underlying protocol is plaintext HTTP requests or something similar), upon receiving a connection from a client, rewrites the destination's response to change all gendered pronouns to those of some other gender. Your sample code should:
Compile and/or run without errors;
Demonstrate an ability to write networking code;
Demonstrate the ability to do text manipulation in a safe and efficient manner;
Show an understanding of how a basic proxy application functions;
It is entirely permissible to use libraries to achieve the goal. For the blocking and asynchronous settings respectively, in Rust one might look at rust-socks or socks5-rs, and pysocks5 or txsocksx for Python.
A brief proposal for how you would implement this project (it's okay to bevague and/or include questions, part of the internship will involve mentoringand continual feedback);
"$LANGUAGE" probably meant "scripting languages", as you said about the common reference. I actually dug around: found no clue. Even asked Larry Page multiple times: no answer! Oops!
And by editing the post, Isis Agora Lovecruft (the hacker, physicist, FBI wanted, among many other things for sure) has just cast another cloud over "what $LANGUAGE meant" again, btw!!! I -- surely am -- not sure what it meant now: other languages similar to Python/Rush or scripting languages or something else. It's w/e anyways! xD
I suspect the answer is "This is about getting the task done, so thinking about it in terms of hours per week is the wrong question. How long will it take you to do the task?"
Yes, arma is correct here. This is basically a small contract to do a task. If you can get it done by working one day per week, that's great! However, it's also perfectly fine if you need to take more time to learn/try things along the way.
When launching Tor, sometimes the IP 199.254.238.52 shows up as ESTABLISHED. I don't know why or how, because the IP is no longer listed anywhere on current lists and local cache. I know 199.254.238.53 exists and is legit, but why is 199.254.238.52 showing up?
But the answer is that longclaw, one of the directory authorities, has the .52 address hard-coded in the Tor client. You can find it in src/or/config.c.
is it relevant to involve a study about censorship/ddosspartner/joint-venture ?
i mean that i access secure service using Tor and the relay look like compromised if the service (e-mail provider e.g.) has a partnership with ddoss protection compagny (israeli/military e.u, nsa) so should it be possible to implement a switch-function balancing/dropping/cutting the connection in case of corruption ?
scanning, balancing, then reporting could be done by a machine as an automatic task ; i do not understand why you need a human being for that job.
is it a static automatic scanner for balancing the bandwidth without any ... counter-measure dropping/blocking/ the bad relay ?
should it show a compromised relay ?
could you use it as a detection tool before the Bandwidth Authority do the measurement ?
is it relevant to black-list the suspicious relays and to drop it of the relays-networks ?
if it is just an announce (job) : create a bridge bandwidth scanner (5 000$) it should be put in another category : Tor recruits ... google summer ... i thing you want the shape/model be built then you will tweak it.
I didn't try it, but according to Wikipedia, "AddressSanitizer does not prevent any uninitialized memory reads..." You'd need something like Valgrind for that I suppose. But who would ever write something like that in the first place? Okay, don't answer that.
So, yes, but presumably not with -Wuninitialized -Wmaybe-uninitialized -Werror.
How do you make the difference between a ghost a virtual or a unknown relay (new or compromised) infiltrating the network during the transfer/connection ?
i suppose there are some label or flag and you can't afford to sort an 'undefined noise' ...
If your challenge is a success ; you will be in front of a big problem : correcting the errors so you need the help of the expert working in the space area ; does it imply a quantum audit ?
Your article _ but i must be wrong _ proposes an ambitious project which looks like programming a stingray but reversed in its genuine function toward ... a better independence of the tor network.
Interestingly, the definition(s) of "data mining" seems yet to converge though. I think the Oxford English Dictionaries' definition is pretty not good: [noun, uncountable, computing] "looking at large amount of information that has been collected on a computer and using it to provide new information". The definition from wikipedia.org is much better, I think.
What does "a quantum audit" mean, btw?? Never heard about that! An audit at the finest level?!
Assuming you got Tor Browser from https://torproject.org and it is the legitimate Tor Browser, this is most likely a false positive from your antivirus software.
if I upload a profile image or other kind of files from my computer to facebook, twitter, etc. on the Tor browser, they CAN TRACE ME using the path of the file on the computer or, simply, capturing my real IP when I upload the image or file?
Most likely not via a leak of your real IP address. But many files have metadata attached to them. For example, there's a lot of metadata attached to photos you take with your phone. If you copy it to your computer and then upload it to Facebook, you've (securely) given Facebook a lot of information from that photo. (Whether or not Facebook strips that metadata before making the photo available for everyone, I don't know. They might. imgur.com does.)
For "the path of the file on the computer" part, Tor Browser has its own directories to download and upload by default. In GNU/Linux, for example, it's "tor-browser_en-US/Browser/Desktop/" and "tor-browser_en-US/Browser/Download/"
That default folder must exist for some reason. I always download to/upload from the default directories, because it solves the issue you've asked regarding "the path of the file on the computer" (I guess).
Sounds like the key at https://fyb.patternsinthevoid.net/isis.html used a more advanced standard, like PGP 2x, and that's why gpg cannot import it without some sort of extension.
hahaha....
I worked! LOOOOOOOOOOOL. I was thinking of that, but couldn't brainstom the... keyserver! LMAO! Hu Da Thug it's "gnu key server" by default?!
Someone still block my connection to keys.gnupg.net. Had to torify the line to obtain the key!
Totally fabulous and... hilarious... at the same time!
The new Tor browser is useless since it does not work with OSX 10.8.5
For those who suggest updating OSX:
Who wants to update to Siri shit and the fact that Sierra does not work with older external DVD player or SD card readers.....
And yet another option is to coordinate with one of the non-profits under the torservers.net umbrella to subsidize exit relays running on your infrastructure.
Comments
Please note that the comment area below has been archived.
I really do hope this gets…
I really do hope this gets implemented, I've seen way too many bridges that tend to become slow over time and frustrate user experience.
Can I ask: What is the…
Can I ask: What is the status of pluggable transports in China? Which ones are known to work by default?
Indeed, this is not for…
Indeed, this is not for those who're NOT in love w/Tor!
The IRC is on irc.oftc.net,…
The IRC is on irc.oftc.net, not .org.
Thanks, fixed.
Thanks, fixed.
Thanks both!
Thanks both!
Are high school students…
Are high school students allowed to apply?
I think so?…
I think so?
I'll let Isis clarify if needed.
Yes, high school students…
Yes, high school students are welcome! I've edited the post to say that it's not necessary to be (or have been) a student to apply.
What does "$LANGUAGE" mean,…
What does "$LANGUAGE" mean, and why the "$" sign, please?
Object-oriented languages like C/C++ (I guess)?!
It is a reference to…
It is a reference to variables in many common scripting languages.
I guess it is also a signal for the sort of person they are hoping to attract for this spot. :)
Thank you arma,…
Thank you arma,
"$LANGUAGE" probably meant "scripting languages", as you said about the common reference. I actually dug around: found no clue. Even asked Larry Page multiple times: no answer! Oops!
Hahaha... I was talking to…
Hahaha... I was talking to Roger Dingledine!
And by editing the post, Isis Agora Lovecruft (the hacker, physicist, FBI wanted, among many other things for sure) has just cast another cloud over "what $LANGUAGE meant" again, btw!!! I -- surely am -- not sure what it meant now: other languages similar to Python/Rush or scripting languages or something else. It's w/e anyways! xD
Is it possible for non…
Is it possible for non-students (e.g. graduated in 2015) to apply? Thanks
I think whether you're a…
I think whether you're a student or not doesn't matter for this spot.
Right! Non-students and/or…
Right! Non-students and/or ex-students are welcome!
Is this for students only?…
Is this for students only? any deadline for applying?
Not for students only, I…
Not for students only, I believe.
Great question about the deadline -- I'll wait for Isis to answer that one.
The deadline is 26 June, I…
The deadline is 26 June, I've edited the post to mention that.
How many hours per week are…
How many hours per week are we expected to work on this?
I suspect the answer is …
I suspect the answer is "This is about getting the task done, so thinking about it in terms of hours per week is the wrong question. How long will it take you to do the task?"
Yes, arma is correct here…
Yes, arma is correct here. This is basically a small contract to do a task. If you can get it done by working one day per week, that's great! However, it's also perfectly fine if you need to take more time to learn/try things along the way.
When launching Tor,…
When launching Tor, sometimes the IP 199.254.238.52 shows up as ESTABLISHED. I don't know why or how, because the IP is no longer listed anywhere on current lists and local cache. I know 199.254.238.53 exists and is legit, but why is 199.254.238.52 showing up?
You are really off-topic for…
You are really off-topic for this post.
But the answer is that longclaw, one of the directory authorities, has the .52 address hard-coded in the Tor client. You can find it in src/or/config.c.
"You are really off-topic…
"You are really off-topic for this post."
Yet you kindly responded with the solution. Thank you.
is it relevant to involve a…
is it relevant to involve a study about censorship/ddosspartner/joint-venture ?
i mean that i access secure service using Tor and the relay look like compromised if the service (e-mail provider e.g.) has a partnership with ddoss protection compagny (israeli/military e.u, nsa) so should it be possible to implement a switch-function balancing/dropping/cutting the connection in case of corruption ?
I'm not sure I understand…
I'm not sure I understand what you're asking. Could you please try re-phrasing the question?
scanning, balancing, then…
scanning, balancing, then reporting could be done by a machine as an automatic task ; i do not understand why you need a human being for that job.
is it a static automatic scanner for balancing the bandwidth without any ... counter-measure dropping/blocking/ the bad relay ?
should it show a compromised relay ?
could you use it as a detection tool before the Bandwidth Authority do the measurement ?
is it relevant to black-list the suspicious relays and to drop it of the relays-networks ?
if it is just an announce (job) : create a bridge bandwidth scanner (5 000$) it should be put in another category : Tor recruits ... google summer ... i thing you want the shape/model be built then you will tweak it.
Yes, this post is looking…
Yes, this post is looking for a human being who will build an automated bandwidth measurement system. ?
I don't suppose C with…
I don't suppose C with AddressSanitizer counts as memory safe?
Wouldn't the following…
Wouldn't the following unsafe code still compile and run with -fsanitize=address?
void main() {
int x;
printf("%d", x);
}
I didn't try it, but…
I didn't try it, but according to Wikipedia, "AddressSanitizer does not prevent any uninitialized memory reads..." You'd need something like Valgrind for that I suppose. But who would ever write something like that in the first place? Okay, don't answer that.
So, yes, but presumably not with -Wuninitialized -Wmaybe-uninitialized -Werror.
How do you make the…
How do you make the difference between a ghost a virtual or a unknown relay (new or compromised) infiltrating the network during the transfer/connection ?
i suppose there are some label or flag and you can't afford to sort an 'undefined noise' ...
If your challenge is a success ; you will be in front of a big problem : correcting the errors so you need the help of the expert working in the space area ; does it imply a quantum audit ?
Your article _ but i must be wrong _ proposes an ambitious project which looks like programming a stingray but reversed in its genuine function toward ... a better independence of the tor network.
Didn't you ask for some sort…
Didn't you ask for some sort of "data mining"?!
Interestingly, the definition(s) of "data mining" seems yet to converge though. I think the Oxford English Dictionaries' definition is pretty not good: [noun, uncountable, computing] "looking at large amount of information that has been collected on a computer and using it to provide new information". The definition from wikipedia.org is much better, I think.
What does "a quantum audit" mean, btw?? Never heard about that! An audit at the finest level?!
Serious security issue …
Serious security issue, Keylogger detected in TOR BROWSER xul.dll
Assuming you got Tor Browser…
Assuming you got Tor Browser from https://torproject.org and it is the legitimate Tor Browser, this is most likely a false positive from your antivirus software.
are students are allowed to…
are students are allowed to learn
who knows nothing about programming languages..?
just a question…
just a question
if I upload a profile image or other kind of files from my computer to facebook, twitter, etc. on the Tor browser, they CAN TRACE ME using the path of the file on the computer or, simply, capturing my real IP when I upload the image or file?
thanks
Most likely not via a leak…
Most likely not via a leak of your real IP address. But many files have metadata attached to them. For example, there's a lot of metadata attached to photos you take with your phone. If you copy it to your computer and then upload it to Facebook, you've (securely) given Facebook a lot of information from that photo. (Whether or not Facebook strips that metadata before making the photo available for everyone, I don't know. They might. imgur.com does.)
For "the path of the file on…
For "the path of the file on the computer" part, Tor Browser has its own directories to download and upload by default. In GNU/Linux, for example, it's "tor-browser_en-US/Browser/Desktop/" and "tor-browser_en-US/Browser/Download/"
That default folder must exist for some reason. I always download to/upload from the default directories, because it solves the issue you've asked regarding "the path of the file on the computer" (I guess).
They public key athttps:/…
They public key at
https://fyb.patternsinthevoid.net/A3ADB67A2CDB8B35.tx
was expired and became unusable. The key at
https://fyb.patternsinthevoid.net/isis.html
cannot be imported for some reason:
gpg: no valid OpenPGP data found.
gpg: Total number processed: 0
Sounds like the key at https…
Sounds like the key at https://fyb.patternsinthevoid.net/isis.html used a more advanced standard, like PGP 2x, and that's why gpg cannot import it without some sort of extension.
I wonder if a simple gpg -…
I wonder if a simple
gpg --recv-keys a3adb67a2cdb8b35
would work for anon. It does for me.hahaha…
hahaha....
I worked! LOOOOOOOOOOOL. I was thinking of that, but couldn't brainstom the... keyserver! LMAO! Hu Da Thug it's "gnu key server" by default?!
Someone still block my connection to keys.gnupg.net. Had to torify the line to obtain the key!
Totally fabulous and... hilarious... at the same time!
you should set hkps…
you should set hkps
gpg --keyserver hkps://hkps.pool.sks-keyservers.net --recv-keys
Bummer! The new Tor browser…
What a bummer!
The new Tor browser is useless since it does not work with OSX 10.8.5
For those who suggest updating OSX:
Who wants to update to Siri shit and the fact that Sierra does not work with older external DVD player or SD card readers.....
how can DTS-NET.COM sponsor…
how can DTS-NET.COM sponsor torproject.org
There are plenty of ways!…
There are plenty of ways!
Check out the sponsors list here:
https://www.torproject.org/about/sponsors
And then see also the donor faq:
https://donate.torproject.org/donor-faq.html
In addition, some ISPs and VPS providers donate money to torservers.net so they can run more exit relays:
https://www.torproject.org/docs/faq#RelayDonations
And yet another option is to coordinate with one of the non-profits under the torservers.net umbrella to subsidize exit relays running on your infrastructure.
Thanks for wanting to contribute!