copied file cannot be found by peer

[anarcat]

so @anarcat's ipfs adventures continue! the story so far:

1. i added a 3.2GB file on my laptop, which took an hour (ipfs add horrendously slow #898)
2. i copied it to my workstation, which took an hour, but completed (local transfer takes forever #872)
3. my friend, during that time, tried to copy the file from the laptop, and got around 1GB of the file until i shutdown the laptop

Now the file is on the workstation, but my friend can't get the file. It's still stuck at 1GB, even though a daemon is running on the workstation. I expected this to just work...

[whyrusleeping]

The way I would go about debugging this, is make sure you know the peer ID's of both machines, make sure they are both running, then start the file request.

run ipfs swarm peers and confirm that the other peer is in the list (this means there is an active connection between the two). If the peer does not show up, try a manual connection (if you have to do this, this means the DHT isnt resolving properly):

$ ipfs swarm connect /ip4/$IPADDR/tcp/$PORT/ipfs/$PEERID

If that completes successfuly, bitswap should send its wantlist to the newly connected peer, resulting in a continuation of the transfer, if the transfer resumes after the connection succeeds, then that tells me something is wonky in the DHT.

If both peers are connected directly, and still no transfer is happening (and youre sure the file is actually where you think it is) you should check ipfs bitswap stat.

This command should show a bunch of entries in your wantlist, these should be the blocks of the file you are requesting, in either case (blocks in wantlist or not) i have no idea what would cause that. Getting a stack dump of both machines would help me out a bunch.

[anarcat]

i don't see the other node in ipfs swarm peers. I can't connect using ipfs swarm connect /ipfs/$PEERID. He's probably behind NAT so I don't know his real IP.

I'm not the one wanting the file, so ipfs bitswap stat doesn't show anything in the wantlist.

How do I generate a stack dump?

[whyrusleeping]

you can generate a stack dump with Ctrl + \ (it sends a SIGQUIT).

It he is behind a NAT, then he will have to try doing an ipfs swarm connect to you, If that fails as well, (you are both behind restrictive NATs), then we currently can't make that work. If thats the case, we should start looking at relay connections sooner rather than later.

[anarcat]

we are both behind NAT.

funny thing is that he was able to transfer 1GB of the file while it was on the laptop, which is behind the same NAT! it's only since the laptop is closed that the file can't be transfered.

[anarcat]

in other words, NAT to NAT transfer worked at some point, i believe.

[whyrusleeping]

okay, then I would have him try doing an 'ipfs swarm connect' to you.

[anarcat]

seems like there was some progress now, went from 1.08GB to 1.10GB... so i guess something is happening. maybe it just seems so slow (because of #872) that my friend thought it was just stopped..?

[micah]

It is definitely possible that I thought that it just stopped, it is hard to see progress when it is updating only at 1GB at a time. However, I was pretty sure that I had left it for a long time without it progressing at all (8hrs at minimum). I dont know if @anarcat was in my peer list then, but he is now.

[whyrusleeping]

okay, slow is something that will likely be fixed by #900, if not... Ill have to debug some more!

[jbenet]

I suspect there may be dht problems at play here. If there's an intermediate node that's our clients cant find they'll just stall. (since the dht isnt yet huge, it still suffers from nodes going up and down or our still imperfect NAT traversal). Though this stalling should not happen (even with dht problems) if you're directly connected. bitswap can bypass routing entirely if two nodes are directly connected and making forward progress.

@whyrusleeping maybe we should have something like a manual "ipfs routing provide -r "

[whyrusleeping]

Going to close this for now, feel free to reopen if the problem persists or reappears