Follow

A customer asked for a cost-effective solution to share a 10 gig file to upwards of 3,000 users per month, with a reliable uptime and minimal outage.

They proposed something like Amazon or Azure.

Rubbish.

A cost-effective solution to make a large file available to a vast number of people is BitTorrent. It's been working for almost two decades, perfectly, despite numerous attempts to shut it down.

Why do they try to shut it down?

Because it's too honking good at what it does, and some people don't like it.

@drq
bittorrent's applicability depends entirely on usage patterns. For example, if most people happen to shut down their system at particular time then the file can become unavailable or download speed will be slow.

@loonycyborg Over the numbers as large as 3,000 users per month it won't be all that noticable.

Besides, you can setup s couple of VPS or dedicated servers to seed constantly which the system can fall back to.

@drq
Yes, they will be slow, it will require discipline for users, etc. They need to both download and actually remember to run torrent program. 10Gig you can get for free in some file hosting services iirc. Torrent might be better if you want to share 1 terabyte of very specific data instead, with people who most definitely will download and share it.

@loonycyborg

> Yes, they will be slow, it will require discipline for users, etc. They need to both download and actually remember to run torrent program.

Inb4 github.com/webtorrent/webtorre

@loonycyborg @drq Genuinely curious: In what case (if any) would a torrent with a web seed be slower than direct downloads from that same server?

@loonycyborg @drq But if you, as I said, have a web seed, and none of the users is seeding, everyone is effectively downloading from the web seed, why would that be slower than direct downloads?

@L12C @drq
It it won't be then there's no difference with plain http/ftp download.

@loonycyborg So we already have our baseline, right? And as more people download it, the more system scales. The only way from here is up.

@L12C

@drq @L12C
In that case there is no reason to offer anything other than http/ftp. Torrent would pay off only in case there isn't enough bandwidth on single server. But in that case issues of seeder discipline apply.

@loonycyborg Well, the reason, as I said, is 30000 uses per day.

With HTTP download, the more users there is, the worse off one individual user is.

With torrent, quite the opposite.

@L12C

@drq @L12C
Theoretically it's true but actual usage patterns totally could throw a wrench here. All depends on how long seeds remain active. Nowadays even 1 Gb download can be really fast and if most of them do one download and then go offline then it will work no better than one http server.

@loonycyborg @drq @L12C $2900 per month without TAXES for 30 000 GB (10 Gigabytes * 3000 users) outgoing traffic from S3?!
calculator.s3.amazonaws.com/in

With Azure will be the same (and with Google cloud too).

I could not call this "cost effective". I lean over to "definitely insanity".

@_1751015 @drq @L12C
And how much will it be with actual file hosting service like dropbox?

@loonycyborg @drq @L12C Dropbox Business Advanced with "Dropbox Transfer" gives ability to send up to 100GB files even to users that don't have Dropbox accounts. The price is 3 licences * 15 Euro per month = 45 Euro/month without taxes.
I don't know how fast is the downloading for 10GB files.

@_1751015 @drq @L12C
Currently I'm using mostly pcloud and their free tier is 10Gb storage and it seems you can make links even on free tiers.

@loonycyborg @_1751015 @drq You can make links but as you can see on pcloud.com/de/cloud-storage-pr you have traffic limits that make distributing large files to 3000 people per month beyond unreasonable

@L12C @loonycyborg @drq So the file sharing cloud syncing services are not applicable.
Lets check the old school solution: 3-5 dedicated servers around the globe with 1Gbps guaranteed networking, rsync scripts, domain name, DNS service and 2 hours/week for support/maintenance.
In Europe you can get unmetered guaranteed 2 Gbps with dedicated server for around $250 per month (ovh.com/world/dedicated-server). I guess for USA and South America it will be the same. I don't know about Asia-Pacific region and Africa.
So:
1) 3 servers x 250 = $750 per month, 5 servers = $1250-1500
2) domain and DNS let's say 20 per month
3) 2 hours per week per one person - depends but with each solution it will be roughly the same

@_1751015 @loonycyborg @drq The problem is that Dropbox pricing is based on storage volume, not traffic volume, but they have been known to temporarily block accounts that cause "too much" traffic. According to dropbox.com/help/security/bann that limit is 200 GB per day for paid accounts, so as soon as more than 20 people try to download your 10 GB file in one day you're screwed.

@drq @loonycyborg
the strength of BitTorrent isn't in making files available any time someone demands them.

It's in decreasing the load on the server when multiple users download in parallel.

If you have only 1 seed and 3000 users downloading simultaneously, the seed will only upload each piece once.

Now if they all finish downloading, go away, and another 3000 users come, then the seed will need to upload it again. But it's still 3000x amplification factor.

@wolf480pl
> It's in decreasing the load on the server when multiple users download in parallel.

Yeah. As in, "being cost-effective".

@loonycyborg

@drq @loonycyborg
exactly.

My point is, you don't really need users to seed after they finish downloading. Sure, it helps, but even if they don't, it's still N times better than http, N being the number of simultaneous connections.

@wolf480pl @drq
And if it's a small file like 1Gb it's unlikely that they'll coincide.

@loonycyborg @drq
oh, it's 3000 users per *month*
that's like, 10 users per day...
yeah, unlikely.

@blobyoumu Piracy existed before BitTorrent, and will exist after BitTorrent.

Besides, piracy these days is just a code word for exactly that - copying large files across a large number of people. Guess what's BitTorrent for, and what it's good at?

@drq it's not a cause but a solution, i know it.
Just pointing why companies don't want to use it.

@blobyoumu Yeah. Because they're idiots with buzzwords for brains.

@drq >not delivering flash drives to people by mail
@drq a certain company with their game launcher are using BitTorrent for client downloading, so it's not exactly that nobody uses it.

@drq Also notice that many companies (including my employer, unfortunately) are stupid and filter everything that they don't consider 100% work-related.

@drq
Sharing with BitTorrent is almost free, unless you dedicate multiple servers with large bandwidth around the globe

Free is not capitalist, with no or low margin profit for IT business

@drq as far as I know it was also used (maybe is still used?) in delivery of World of Warcraft and Star Trek Online. Linux distributions used to rely on bit torrent as well. It works!

@drq I don't think they use the bittorrent protocol specifically, but even Microsoft is using peer-to-peer file transfers - in *Windows Update*.

I suspect the main reason is so multiple computers in a LAN don't have to download everything from MS servers separately, but it also (you can turn this off) shares to machines on the internet. And since Windows Update is an always-on background service, it can "seed" without user intervention.

@drq Do you need restricted access for that file? Is there a simple solution for that with torrents?

@threed @drq Own tracker and not given everyone the torrent file.
Of course people can share the torrent. But they can share the downloaded file as well.

Sign in to participate in the conversation
Mastodon.ml

Русская нода социальной сети "Мастодонт", части Fediverse - всемирной федерации социальных сетей. Зона общения, свободная от рекламы и шпионажа, теперь и в России.