silentct/internal/monitor, branch mainMirror of https://git.glasklar.is/rgdd/silentcthttps://git.rgdd.se/silentct/atom?h=main2025-10-05T07:05:28+00:00monitor: Retry get-sth and proof fetching2025-10-05T07:05:28+00:00Rasmus Dahlbergrgdd@glasklarteknik.se2025-05-11T18:54:16+00:00urn:sha1:fcd0a8252e3ab27662be2a781073b80a03f31573
Should ensure we don't get into a position where we always fail to get
3x queries that succeed in a row when trying to persist chunks.
fix: Ensure rate-limits are on for get-entries2025-01-06T12:30:52+00:00Rasmus Dahlbergrgdd@glasklarteknik.se2025-01-06T12:21:47+00:00urn:sha1:2d3b1f2cb0c05385c1702f1a7d74fa08d52c262f
Backoff on 4XX and 5XX. See related issue:
https://github.com/google/certificate-transparency-go/issues/898
Test manually hints:
```
$ cat srv.py
from http.server import HTTPServer, BaseHTTPRequestHandler
class RequestHandler(BaseHTTPRequestHandler):
def do_GET(self):
self.send_response(429)
self.send_header("Content-Type", "text/plain")
self.end_headers()
self.wfile.write(b"429 something something...")
def do_POST(self):
self.do_GET()
def do_PUT(self):
self.do_GET()
def do_DELETE(self):
self.do_GET()
if __name__ == "__main__":
server_address = ('localhost', 9090)
httpd = HTTPServer(server_address, RequestHandler)
print("Server running on http://localhost:9090")
httpd.serve_forever()
```
And a transport for http.Client that redirects to localhost:
```
type statusRR struct {
inner http.RoundTripper
}
func (s *statusRR) RoundTrip(req *http.Request) (*http.Response, error) {
if strings.Contains(req.URL.Path, "ct/v1/get-entries") {
req.URL.Scheme = "http"
req.URL.Host = "localhost:9090"
}
rsp, err := s.inner.RoundTrip(req)
return rsp, err
}
```
fix: Ensure backoff for get-sth and proof fetching2025-01-05T13:59:09+00:00Rasmus Dahlbergrgdd@glasklarteknik.se2025-01-05T13:59:09+00:00urn:sha1:ee5c55d82ad70353e8203f6729d4325b59741bfa
Our get-entries fetcher already backs-off exponentially.
fix: Ensure fresh STHs are propagated2025-01-05T13:34:04+00:00Rasmus Dahlbergrgdd@glasklarteknik.se2025-01-05T13:34:04+00:00urn:sha1:b8b6ce265ef083d21db155af4a50cee1b9b0b934fix: Don't accept timestamps that shrink2025-01-05T13:34:04+00:00Rasmus Dahlbergrgdd@glasklarteknik.se2025-01-05T13:34:04+00:00urn:sha1:a7cc687c7bab91b9e525d9b91dc9d358afca5a28fix: Ensure chunks are sent eventually and on exit2025-01-05T13:34:04+00:00Rasmus Dahlbergrgdd@glasklarteknik.se2025-01-05T13:34:04+00:00urn:sha1:108932cab926cf9743df7f9eaf383104130e9377chore: Remove dash in project name2024-06-02T06:36:35+00:00Rasmus Dahlbergrasmus@rgdd.se2024-06-02T06:29:49+00:00urn:sha1:279de6a1195adb739a8d1f2afb445b68793b28b8
To be consistent with naming of the tools in cmd/.
Only bootstrap a compact range once per log2024-05-26T13:37:58+00:00Rasmus Dahlbergrasmus@rgdd.se2024-05-26T13:37:58+00:00urn:sha1:20f52e16880210b1893d89e2d20819171632da32
As opposed to doing a new bootstrap with get-proof-by-hash every time
the next root is constructed. Bootstrapping the compact range from a
get-proof-by-hash query works for the most part, but fails if the log
included a duplicate entry and gives us the index for that instead. Log
operators with duplicate entries include Cloudflare and Digicert.
If bootstrap fails (unlucky), we try to bootstrap again once the log's
signed tree head moved forward (hoping the last entry has no duplicate).
The more reliable way to bootstrap a compact range would be to use the
get-entry-and-proof endpoint. This does not work in practise because
some logs are not implementing this endpoint. Digicert has such logs.
Reduce default chunk size2024-05-16T06:51:38+00:00Rasmus Dahlbergrgdd@glasklarteknik.se2024-05-16T06:51:38+00:00urn:sha1:6567c8f4ec3eefb855c6ef630a53b0fb8d8bf1e9Fix nits spotted by go vet2024-05-16T05:47:04+00:00Rasmus Dahlbergrasmus@rgdd.se2024-05-16T05:47:04+00:00urn:sha1:bea0a1bce37a73acb323269edeaa5b7d11a2b42d