The Wilmington Watch: A Tor Network Team Hackfest
The Tor network team is a small team responsible for developing the core Tor daemon. We're located around the globe, so we periodically meet in person for team hackfests to keep our team fresh and up to date with all things Tor, and to fast-track features and improvements. Previously, we've met for hackfests in Arlington and Montreal, and for our latest meeting, we met in Wilmington, Delaware, a town revered for its yearly Italian Festival.
To better understand the geographical setting of this meeting, go to Trenton, New Jersey, USA, hop into a Delaware river riverboat, then follow the flow south; after about 60 miles you will eventually see a town named Wilmington on your right. This small town hosted us for a few days while we worked on making Tor stronger and safer.
What went down
We worked intensely for several days and nights, researching, planning, and cooking meals for each other. Here is a small fraction of the topics we worked on:
-
We schemed on about the Tor modularization project which aims to clean up and organize our codebase into nice and tidy abstract modules. Cleaning and modularizing our code not only reduces technical debt, but also allows us to eventually rewrite those submodules into higher-level languages such as Rust, D or APL.
- That last part is not science-fiction; as of a month ago, Tor's build system is capable of building and linking code modules written in Rust. We are now actively working on implementing the protover subsystem into Rust as our first prototype module.
- As part of the security discussion, we talked about the new padding defenses that were recently added to Tor and provide cover to Tor circuits against traffic analysis. We made plans for future padding techniques and defenses.
-
We also briefed up the whole team on how our new entry guard picking algorithm works to enhance the security of Tor clients and protects them against local network attacks. We planned various defenses against hidden service guard discovery attacks, as well as alternative onion routing path algorithms. Our next step for improving guard security is to simulate alternative path construction algorithms and evaluate their performance and security guarantees.
- We discussed KIST, an alternative network scheduler and congestion management logic for Tor which offers improved circuit performance and cleaner network tubes. KIST is currently under active development, so we roadmapped and planned for how to get it included upstream.
- We talked about our code testing techniques and how to improve them. We discussed the necessity of regression tests, the need to improve our integration tests, and also the future of our fuzzing framework. (Feel free to get in touch if you want to help improve our tests!)
-
We all shared our experiences and thoughts on our beloved tool for ticket tracking and project management: Trac. We discussed ways we could improve our Trac workflows and also alternative tools we could potentially try (e.g. Gitlab). Transitioning to another tool is not so easy, though; since our Trac instance contains 10+ years of Tor history, we need to make sure we don't lose any information.
We stayed for about 5 days in town doing all these things and more. Then on a Friday afternoon as the Wilmington Italian Festival was setting up for yet another day, we jumped on trains, planes, and buses and moved on to other places and stories. Life goes on, and the same goes for Tor development.
We're committed to being open and transparent about our work, and we hope you enjoyed this post. Keep on hacking.
Sponsor your own hackfest
If you find these sort of hackfests exciting, and you would like to host or sponsor one, don't hesitate to get in touch us at press@torproject.org with "Hackfest" in the subject line.
Comments
Please note that the comment area below has been archived.
Uhhh... I couldn't figure…
Uhhh... I couldn't figure out how Nelson Mandela and Alien Abduction pictures fit into the theme of the post. xD
The Nelson Mandela paintings…
The Nelson Mandela paintings were on the wall in the room where we did the brainstorming and writing-on-pieces-of-paper. So when we moved to a new piece of paper, we put the old one up on the wall, and of course you need to put them up *around* the paintings, not over them, to make sure not to damage them.
Alien Abduction, on the other hand, was one of the rides at the Italian festival that people went to one of the evenings.
Ahhhh... I hope many Chinese…
Ahhhh... I hope many Chinese cyber-strategists will be reading this post, so they may be aware of which kind of person they are facing in "the Sino-Tor war over the Great Firewall". xD
Don't forget a white-plastic board + marker pens of different colors. I find that extremely useful in group meetings of discussion/study. xD
We discussed KIST, an…
Interesting... in practice will it have a significant impact on Tor's speed?
Hey, I don't know the…
Hey, I don't know the precise values here but KIST indeed helps with congestion management on relays, that is relays that are overloaded will schedule their circuit handling better.
Also, KIST is a simplified version of our currently circuit scheduler, which will make our codebase cleaner and more readable. Which is another plus.
KIST prioritizes "web"…
KIST prioritizes "web" circuits (low bandwidth usage, traffic only periodically in small bursts) over "bulk" circuits (high bandwidth usage, constant traffic). When a relay is choosing which circuit to pull cells from next to send out into the network, with KIST it will pick "web" circuits before "bulk" ones. For more information on Tor's shortcomings (as of a few years ago) and how KIST works, see the KIST paper.
I am cautiously optimistic based on our large scale simulation results. Long live network tests are wrapping up. I expect to be able to update the KIST trac tickets soon.
Thanks a lot pastly for the…
Thanks a lot pastly for the non-technical breakdown!
The safest way to obtain Tor…
The safest way to obtain Tor Browser still appears to be to download and verify the tarballs from www.torproject.org, using a previous TB (or Tails). Would KIST discriminate against downloading the latest TB tarball--- or even worse, the latest Tails iso image?
In the experiments conducted…
In the experiments conducted so far (and there have been a ton...) I have not seen "bulk" downloads getting slowed down.
While it is true on the micro-scale of individual scheduling decisions (of which there are 100 per second) that "web" circuits will be written to the kernel first, it appears that on the macro scale of time-to-last-byte that "bulk" circuits are not any slower. Part of the reason why could be that "bulk" circuits want to be scheduled every single time but "web" circuits only want scheduling every now and then. There may be plenty of times when there are only "bulk" circuits that need scheduling. KIST also seems to be able to better utilize the available network capacity, meaning KIST might make the Tor network a little faster (this is not a claim I'm comfortable making strongly at this time). That could also explain why "bulk" circuits don't seem to be slowed: they are being adversely affected by the prioritization but a faster Tor balances that out.
Just two theories.
In short, I don't predict large downloads like a Tor tarball or a Tails iso taking much longer, if longer at all.
How will the transition to…
How will the transition to Rust affect the ability to compile from source?
Initially, Rust support will…
Initially, Rust support will be optional (like now) so not much will change.
In the future, if Rust becomes more widespread and Tor ends up using it more, we might really bake it into Tor, and at that point you will need the Rust toolkit to compile Tor. I think we are pretty far from this goal right now tho.
Can you tell me if a TVFrog…
Can you tell me if a TVFrog would do his work through a TOR-circuit in China and can so receive ALL the TV's-programs in the world for FREE, Netflix, etc already don't like the small apparate from $25-$70.
Thanks for answering, ps they do it by "streaming" the internet, so no more cable-tv.
Side note. For the sake of…
Side note. For the sake of being "FREE": if you take it from "FREEDOM", that would always be great; if you take it to be "priceless", then you will probably crash into many problems.
Nothing "free", freedom is not "free"!
Hello asn, great write up as…
Hello asn, great write up as always! One question concerning Rust: Since AFAIU it provides parallelism, would one easy shortcut to have Tor be fully multi-threaded is to, as an example, make a relay crypto module in Rust?
Hello! While Rust does allow…
Hello! While Rust does allow one to write memory-safe code in a concurrent manner, it doesn't automatically provide it. There's a really good blog post on rust-lang.org that explains how Rust's concept of ownership and memory-safety ties into paradigms for writing concurrent programs. For Tor's case, however, there's some (likely large) software architectural changes needed in Tor's codebase to be ready for being more highly-parallelisable. If anything, ownership and borrowing in Rust will make it easier moving forward to expand upon, and make more safe, Tor's message-passing/worker-pool multi-threading design.
go read wikileaks so many…
go read wikileaks so many exploits
Why Nelson Mandela? He was…
Why Nelson Mandela? He was not peaceful, nor was he a statesman. He was a tyrant to turned to violence, including against non-Communist black citizens of South Africa. There are better historical figures to consider than a man who embraced violence and used it to kill children and other black Africans.
Those pictures were already…
Those pictures were already hanging on the wall in the meeting place.
padding defenses? According…
padding defenses? According to the FAQ I thought you guys were against padding don't get me wrong though i'm excited that your putting some padding.
https://www.torproject.org/docs/faq.html.en
According to your FAQ
Cover traffic is really expensive. And *every* user needs to be doing it. This adds up to a lot of extra bandwidth cost for our volunteer operators, and they're already pushed to the limit.
You'd need to always be sending traffic, meaning you'd need to always be online. Otherwise, you'd need to be sending end-to-end cover traffic -- not just to the first hop, but all the way to your final destination -- to prevent the adversary from correlating presence of traffic at the destination to times when you're online. What does it mean to send cover traffic to -- and from -- a web server? That is not supported in most protocols.
Even if you *could* send full end-to-end padding between all users and all destinations all the time, you're *still* vulnerable to active attacks that block the padding for a short time at one end and look for patterns later in the path.
Thanks for pointing this…
Thanks for pointing this inconsistency out. I opened a trac ticket about it: https://trac.torproject.org/projects/tor/ticket/22958
Note that it's just netflow…
Note that it's just netflow padding to collapse netflow records, it's not the type of expensive padding that the FAQ addresses, but yes the FAQ should be tweaked a bit.
I have IPV6 disabled on my…
I have IPV6 disabled on my LAN and my router.[1]
Why all of a sudden is Tor using ipv6 exit nodes? I don't see them on check.torproject.org, but I've seen them elsewhere. This behavior is new and I don't understand how IPV6 is being used with Tor when it is disabled on my network.
What and where do I modify ipv6 values in Tor settings somewhere so they aren't used? Especially the line about preferring ipv6 traffic. I don't want any ipv6 used. TIA
[1] In fact, my router cannot use IPV6 technology.
> I don't understand how…
> I don't understand how IPV6 is being used with Tor when it is disabled on my network.
The IP version(s) that your local network supports is irrelevant to the IP version(s) that an exit node's network supports. Remember: Tor transports the bodies of TCP packets. If you connect to your guard with IPv4 and the exit makes a connection to your desired destination with IPv1000, it doesn't matter.
I'm not sure why you think you should disable IPv6. Could you elaborate? And you should leave advanced options like these alone if you have a significant adversary. Changing them will make your behavior look different and a big powerful adversary might be able to notice that.
High Tor,…
High Tor,
A year ago I found possibilities to go into darknet with Your site. I like it. Regarding Your project I would be able to translate into german. I'm no professional hacker ( who will guide me? ) and my possibilities are now to use some time into translating. I'm german, 54 years old and living alone. My present used email is o.pongratz@gmx.de
Tell me if You like me to work with You, please
Regards
Otmar