Rendered at 03:34:46 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
hdgr 2 days ago [-]
Bitmagnet -https://bitmagnet.io/ - does exactly that. I left it running for a few weeks and then stopped the crawler. Didn't expect much, but still somewhat disappointed by the garbage it reeled in.
NegativeLatency 2 days ago [-]
I've had one running for over a year now, it's replaced my usage of regular torrent sites completely, there is a lot of junk, and it gets stale, but it's still a better experience than most of the public trackers out there IMO
qingcharles 2 days ago [-]
Are you running it at home?
I built one with a nice TUI to run on a VPS so I can try and find rare magazine torrents, but Hetzner were upset about it. I need to find it a new home. It was a very good citizen, but it still raised too many flags.
NegativeLatency 2 days ago [-]
yes but with a VPN to my seedbox, all the egress from the docker containers its running in goes through this: https://github.com/passteque/gluetun
the seedbox is through https://www.feralhosting.com used them for over 10 years now and they've been great (shared hosting so I have linuxbrew setup there, but no docker sadly)
k4rli 2 days ago [-]
Runs fine at home. I've indexed 20M+ torrents in last few months running it during the day. With Prowlarr (or similar) it could easily replace other indexers.
fc417fc802 2 days ago [-]
I think generally you have to be very conservative about how you use ultra cheap hosts like hetzner simply due to the economics. Either find a more expensive service that will exert more effort towards discretion or alternatively spend $5 per month on a VPN that's friendly to torrents.
farnsworthfusor 2 days ago [-]
I heard the more money you're paying them the more lenient they are.
For cheap hosts look for ones that allow tor exit nodes if you're looking for ones that allow funny stuff. There are some that allow it for ideological reasons. Look through the hundreds on lowendtalk. On that forum you can even ask the providers directly if they allow it.
NoMoreNicksLeft 2 days ago [-]
Which magazines?
qingcharles 2 days ago [-]
Anything I don't have! Sometimes I'll find a torrent and no seeds/peers and I'll wonder if there is another torrent out there that has the same files in it somewhere that I can find.
The other day it was trying to track down some older High Times issues that were torrented but the torrent is dead. Last night it was a mag titled Films & Filming which I know is scanned, but I can't find anywhere.
NoMoreNicksLeft 2 days ago [-]
High Times is mostly on archive.org, if you need that one. I'd sort of like the film-making one, I'll put some time into that. On my list of periodicals, I think the count's up to 500 that I consider important enough to archive and I'm nowhere near done with it.
qingcharles 2 days ago [-]
Yeah, I have the High Times from archive.org, but it's missing a lot of issues which were torrented at some point. If there's anything you need send me an email on my profile.
toomuchtodo 2 days ago [-]
Can I get a copy for the Internet Archive? Will take as much of the corpus as you’re willing to provide.
(no affiliation with them)
qingcharles 2 days ago [-]
I'm starting to upload everything very soon. I have >4million magazines here so far I think. Feel free to email me on my profile :)
HugoTea 2 days ago [-]
Do you have old 2000s era Argos catalogues?
Argos is a shop in the UK, I've been tracking down old catalogues to see if my childhood CD player is in there, can't find evidence of it anywhere and so many were made.
The catalogues are sort of rare but were given away for free, now people sell old ones on ebay
qingcharles 24 hours ago [-]
I don't, but I've started a separate project to try and collect as many catalogues as possible, Argos included. I'm after a Maplin catalogue from my youth :) I do remember coming across a site with a bunch of Argos catalogues on it, but I suspect if you've done any searching you came across that site already?
plusfour 2 days ago [-]
Same here, for over a year. how many torrents has yours indexed?
NegativeLatency 2 days ago [-]
28M
plusfour 1 days ago [-]
27. it's creating about 20k a day.
gonzalohm 2 days ago [-]
How much space do you need to store the index?
plusfour 2 days ago [-]
160gb for 27m torrents
knowaveragejoe 2 days ago [-]
The factor that determines this most is how much of the files metadata for each torrent you choose to keep, which is configurable.
drdexebtjl 2 days ago [-]
I disabled mine because it was constantly writing to my SSD.
Modified3019 2 days ago [-]
In case you haven’t considered, HDDs work great for this, since write wear out isn’t really a factor for those. ZFS can also help batch together TXG’s until they hit time or size limits before commit it at once to the disk.
felooboolooomba 2 days ago [-]
I solved it by storing the data on /dev/null
futune 2 days ago [-]
If /dev/null is fast and web scale, I will use it.
Daviey 2 days ago [-]
The writes are insanely fast.
spwa4 2 days ago [-]
The reads are even faster.
permalac 2 days ago [-]
Pretty big space.
muyuu 2 days ago [-]
probably worth adding some ML filter to it because yeah, most of the bulky stuff in bittorrent is always going to be garbage - a lot like the internet generally, the value is in filtering the good stuff out
infinite_spin 2 days ago [-]
out of curiosity, what kind of junk/garbage is typical?
yeeeloit 2 days ago [-]
ungodly amounts of porn.
2 days ago [-]
qingcharles 2 days ago [-]
This is a good tip, thanks. I'll probably replace my home-grown scanner for this one.
naikrovek 2 days ago [-]
why the hell does that require a postgres database instance? has no one heard of sqlite?
"It's just `docker compose up` dude." that's not excuse for unnecessary complexity. software for a single person should be a single binary that anyone can run how they choose. If they want to use Docker, great. If they want to run it in a terminal, great. If they want to run it on a server, great. If they want to run it on their phone, great.
developers: avoid limiting people artificially like this. There is no way that Sqlite is insufficient for this.
knowaveragejoe 2 days ago [-]
The database gets quite large and the developers put in a lot of effort for tuning it for searchability.
gritzko 2 days ago [-]
2010. I remember those times. I was doing these things for science in 2008. Performance-wise, PEX was much faster than DHT. At least, in my setting.
This year, I was giving it as an assignment to students. Does not take much time with LLMs.
I saw mention of Sybil's, hasn't this been patched effectively making techniques of quick discovery though many DHT nodes much more expensive and slower.
You would need a unique IP to overcome this per n nodes. That could be tens of thousands of IPs.
Boss0565 2 days ago [-]
old paper
MoonWalk 2 days ago [-]
The article neglects to define "DHT" before using it.
ivanjermakov 2 days ago [-]
Distributed hash table - ButTorrent extension for discovering torrent's seeders by advertising its hash across known peer pool, think of it as a distributed tracker. Contrary to traditional way of asking a known tracker for peers of that torrent.
Its algorithm is very elegant, using binary search on peers' and torrents' hashes, narrowing down to peers that are more likely to be seeders (or at least know some).
Not a P2P innovation with Bittorrent, FWIW. Kademlia DHT (used in eMule/LimeWire/Gnutella P2P networks) long predates Bittorrent.
MoonWalk 1 days ago [-]
Thanks!
drbscl 1 days ago [-]
> The BitTorrent community has responded to these
trends by deploying decentralizing tracking systems based
on distributed hash tables (DHTs).
No it doesn't. Before you try to correct me, the abstract never counts ;)
2 days ago [-]
MoonWalk 1 days ago [-]
Downvote all you want, but a fact is a fact: The article doesn't define the abbreviation before using it.
dgellow 1 days ago [-]
I'm not sure why you're saying that in the HN comments. As far as I know the authors aren't there, and the paper is from 2010. It's not like it will get updated now.
I built one with a nice TUI to run on a VPS so I can try and find rare magazine torrents, but Hetzner were upset about it. I need to find it a new home. It was a very good citizen, but it still raised too many flags.
the seedbox is through https://www.feralhosting.com used them for over 10 years now and they've been great (shared hosting so I have linuxbrew setup there, but no docker sadly)
For cheap hosts look for ones that allow tor exit nodes if you're looking for ones that allow funny stuff. There are some that allow it for ideological reasons. Look through the hundreds on lowendtalk. On that forum you can even ask the providers directly if they allow it.
The other day it was trying to track down some older High Times issues that were torrented but the torrent is dead. Last night it was a mag titled Films & Filming which I know is scanned, but I can't find anywhere.
(no affiliation with them)
"It's just `docker compose up` dude." that's not excuse for unnecessary complexity. software for a single person should be a single binary that anyone can run how they choose. If they want to use Docker, great. If they want to run it in a terminal, great. If they want to run it on a server, great. If they want to run it on their phone, great.
developers: avoid limiting people artificially like this. There is no way that Sqlite is insufficient for this.
This year, I was giving it as an assignment to students. Does not take much time with LLMs.
https://bittorrent.org/beps/bep_0051.html
You would need a unique IP to overcome this per n nodes. That could be tens of thousands of IPs.
Its algorithm is very elegant, using binary search on peers' and torrents' hashes, narrowing down to peers that are more likely to be seeders (or at least know some).
https://www.bittorrent.org/beps/bep_0005.html
No it doesn't. Before you try to correct me, the abstract never counts ;)
One search tells you what DHT in the context of wikipedia is: https://en.wikipedia.org/wiki/Mainline_DHT