aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorRasmus Dahlberg <rasmus@rgdd.se>2023-03-26 13:25:19 +0200
committerRasmus Dahlberg <rasmus@rgdd.se>2023-03-26 13:25:19 +0200
commit15f5577a2cb2f05977de38eb8572afbff7c875da (patch)
treefff709975c7926c30ea48148f636d42edcd3deb0
parent906527914c2847f2fda763a1ed231cd6e021b1ed (diff)
Add some debug notes
-rw-r--r--README.md13
1 files changed, 13 insertions, 0 deletions
diff --git a/README.md b/README.md
index 25299c8..b28dac4 100644
--- a/README.md
+++ b/README.md
@@ -132,6 +132,19 @@ domains/s, it would be wise to roll-back _at least_ 6000 lines.
Get in touch if you know a fix, e.g., based on `ulimit` and `sysctl` tinkering.
+More debug notes:
+
+ - My system is not fully utilized wrt. CPU/MEM/BW; an odd thing is that it
+ seems to work fine to run multiple onion-grab instances as separate
+ commands, e.g., 3x `-w 280` to get up to ~225 Mbps utilization (max).
+ Adding a fourth instance gets me into the same problem as documented above.
+ - Tinkering with with options in http.Transport doesn't seem help.
+ - Using multiple http.Client doesn't help (e.g., one per worker)
+ - An odd thing is that after errors, it appears that only DNS is dead. E.g.,
+ `curl https://www.rgdd.se` fails while `curl --resolve
+ www.rgdd.se:443:213.164.207.87` succeeds. Replacing the system's DNS with a
+ local unbound process doesn't seem to help though.
+
## Contact
- rasmus (at) rgdd (dot) se