diff options
author | Rasmus Dahlberg <rasmus@rgdd.se> | 2023-03-26 13:25:19 +0200 |
---|---|---|
committer | Rasmus Dahlberg <rasmus@rgdd.se> | 2023-03-26 13:25:19 +0200 |
commit | 15f5577a2cb2f05977de38eb8572afbff7c875da (patch) | |
tree | fff709975c7926c30ea48148f636d42edcd3deb0 | |
parent | 906527914c2847f2fda763a1ed231cd6e021b1ed (diff) |
Add some debug notes
-rw-r--r-- | README.md | 13 |
1 files changed, 13 insertions, 0 deletions
@@ -132,6 +132,19 @@ domains/s, it would be wise to roll-back _at least_ 6000 lines. Get in touch if you know a fix, e.g., based on `ulimit` and `sysctl` tinkering. +More debug notes: + + - My system is not fully utilized wrt. CPU/MEM/BW; an odd thing is that it + seems to work fine to run multiple onion-grab instances as separate + commands, e.g., 3x `-w 280` to get up to ~225 Mbps utilization (max). + Adding a fourth instance gets me into the same problem as documented above. + - Tinkering with with options in http.Transport doesn't seem help. + - Using multiple http.Client doesn't help (e.g., one per worker) + - An odd thing is that after errors, it appears that only DNS is dead. E.g., + `curl https://www.rgdd.se` fails while `curl --resolve + www.rgdd.se:443:213.164.207.87` succeeds. Replacing the system's DNS with a + local unbound process doesn't seem to help though. + ## Contact - rasmus (at) rgdd (dot) se |