aboutsummaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Finalize NEWS v0.1.0v0.1.0mainRasmus Dahlberg2025-03-121-1/+1
|
* README: Shorten downRasmus Dahlberg2025-03-121-6/+2
| | | | | How long it takes ages poorly since rate limit configurations change; and it is hard to tell if we encountered any 429s, see timeline [9].
* timeline: Update with final dataset linkRasmus Dahlberg2025-03-121-1/+2
|
* Add NEWS entry for v0.1.XRasmus Dahlberg2025-03-121-0/+17
|
* chore: Set go module name rgdd.se/ct-sansRasmus Dahlberg2025-03-126-10/+19
|
* Add skipping logs in comments (for debug)Rasmus Dahlberg2025-03-121-0/+4
| | | | Also maybe helpful for others as a quick pointer.
* Add debug print in comments (backoff)Rasmus Dahlberg2025-03-121-0/+1
| | | | Make it easier for others to do similar debugging as me.
* chore: Bump ct library versionRasmus Dahlberg2025-03-122-9/+9
| | | | Ensures we're using the same as silentct, that's good.
* chore: Bump trillian version (and Go version)Rasmus Dahlberg2025-03-122-155/+34
|
* chore: Bump log list dependencyRasmus Dahlberg2025-03-122-3/+3
|
* chore: TidyRasmus Dahlberg2025-03-122-3/+9
| | | | $ go mod tidy
* timeline: Add rate-limit bugRasmus Dahlberg2025-03-121-0/+25
|
* Update manually tuned worker counters for todayRasmus Dahlberg2025-03-121-7/+7
|
* fix: Ensure rate-limits work for get-entriesRasmus Dahlberg2025-03-122-1/+57
| | | | | | | | See further details here: https://github.com/google/certificate-transparency-go/issues/898 We're doing the same fix as silentct until upstream is fixed: https://git.glasklar.is/rgdd/silentct/-/blob/main/internal/monitor/backoff.go
* fix: Update s/Trust Asia/TrustAsiaRasmus Dahlberg2025-03-121-1/+1
| | | | | Description changed in Google's log list. All other log operator names are still unchanged.
* README: Fix irc/matrix linksRasmus Dahlberg2025-03-121-3/+3
|
* README: Add note about time to downloadRasmus Dahlberg2025-03-121-1/+4
| | | | | | | | The sweet spot to take "all current logs" is right after the last year's log shards expired and were pulled from Google Chrome's list. Because then we don't have 12 months of mostly expired certificates to download. Our measurement was a few months after the sweet spot.
* README: No plans on moving source code againRasmus Dahlberg2025-03-121-1/+1
|
* timeline: Add reason for aborting more measurementsRasmus Dahlberg2024-02-191-0/+8
|
* timeline: Snapshot and start collectingRasmus Dahlberg2024-02-101-0/+6
|
* Add link to datasetRasmus Dahlberg2023-04-131-1/+3
|
* s/Quick start/quickstartRasmus Dahlberg2023-04-071-2/+1
|
* Add list all options to READMERasmus Dahlberg2023-04-071-0/+4
|
* Move link to docs/operations.md to the bottomRasmus Dahlberg2023-04-071-2/+5
|
* Add operations timelineRasmus Dahlberg2023-04-072-15/+244
|
* Editorial editsv0.0.2Rasmus Dahlberg2023-03-231-9/+9
|
* Add proper README for the ct-sans tool onlyRasmus Dahlberg2023-03-231-154/+123
|
* Drop debug printRasmus Dahlberg2023-03-231-1/+0
|
* Automate handling of notice fileRasmus Dahlberg2023-03-233-28/+54
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Here's a hacky tool to migrate our ongoing v0.0.1 measurement once it's done. I.e., just split-up the NOTICE prints we have in collect.stdout, putting them in per-log notice files that happens automatically now. ``` // Package main provides a hacky tool that extracts NOTICE: <log desc> prints // from a file collect.stdout, putting them in the logs data directories as // notice.txt. Only meant to migrate away from v0.0.1 that did not store // per-log notice files automatically, which makes things less error-prone. package main import ( "bytes" "encoding/json" "fmt" logger "log" "os" "strings" "gitlab.torproject.org/rgdd/ct/pkg/metadata" ) func main() { directory := "../data" logDirectory := fmt.Sprintf("%s/logs", directory) noticeFile := "../collect.stdout" b, err := os.ReadFile(fmt.Sprintf("%s/metadata.json", directory)) if err != nil { logger.Fatal(err) } var md metadata.Metadata if err := json.Unmarshal(b, &md); err != nil { logger.Fatal(err) } if b, err = os.ReadFile(noticeFile); err != nil { logger.Fatal(err) } lines := bytes.Split(b, []byte("\n")) for _, log := range logs(md) { id, _ := log.Key.ID() desc := *log.Description var notes []byte var numNotes int for _, line := range lines[:len(lines)-1] { if strings.Contains(string(line), fmt.Sprintf("NOTICE: %s", desc)) { notes = append(notes, line...) notes = append(notes, []byte("\n")...) numNotes += 1 } } if len(notes) == 0 { logger.Printf("%s: no notices", desc) continue } logger.Printf("%s: %d notices", desc, numNotes) if err := os.WriteFile(fmt.Sprintf("%s/%x/notice.txt", logDirectory, id[:]), notes, 0644); err != nil { logger.Fatal(err) } } } func logs(md metadata.Metadata) (logs []metadata.Log) { for _, operators := range md.Operators { for _, log := range operators.Logs { if log.Description == nil { logger.Printf("WARNING: skipping log without description") continue } if log.State == nil { continue // skip logs with unknown states } if log.State.Name == metadata.LogStatePending { continue // pending logs do not count towards CT-compliance } if log.State.Name == metadata.LogStateRetired { continue // retired logs are not necessarily reachable } if log.State.Name == metadata.LogStateRejected { continue // rejected logs do not count towards CT-compliance } logs = append(logs, log) } } return } ```
* Drop sanitize of SANsRasmus Dahlberg2023-03-233-179/+18
| | | | Less complex, we will just pass lines to Go's HTTP GET as is.
* Change default to /tmpRasmus Dahlberg2023-03-201-2/+2
|
* Add drafty assemble commandRasmus Dahlberg2023-03-203-6/+316
|
* Fork code snippets to sanitize DNS names (ascii)Rasmus Dahlberg2023-03-201-0/+85
|
* Smaller default paramsRasmus Dahlberg2023-03-181-4/+4
|
* Add notes from starting experimentRasmus Dahlberg2023-03-181-0/+43
|
* Manually add version+logID to STH structv0.0.1Rasmus Dahlberg2023-03-181-4/+2
| | | | Will probably make the data set manifest simpler.
* Light refactoringRasmus Dahlberg2023-03-181-5/+5
|
* Tweak default parametersRasmus Dahlberg2023-03-181-4/+4
| | | | Will put exactly how we run on our machine in README
* Print stats on shutdownRasmus Dahlberg2023-03-182-11/+16
|
* Fix few debug printsRasmus Dahlberg2023-03-182-4/+3
|
* Add one more max workersRasmus Dahlberg2023-03-181-0/+3
|
* Tune number of workersRasmus Dahlberg2023-03-184-5/+31
|
* remove temp fileRasmus Dahlberg2023-03-181-146/+0
|
* check that sans don't contain new linesRasmus Dahlberg2023-03-182-2/+153
|
* rename default dirRasmus Dahlberg2023-03-182-3/+3
|
* Move helpers to separate fileRasmus Dahlberg2023-03-182-38/+47
|
* Print metrics based on flag duration to stderrRasmus Dahlberg2023-03-184-10/+20
|
* update http headerRasmus Dahlberg2023-03-181-2/+2
|
* add creditRasmus Dahlberg2023-03-181-0/+3
|
* renaming files and moving aroundRasmus Dahlberg2023-03-185-33/+42
|
generated by cgit v1.2.3 (git 2.39.1) at 2025-04-02 22:18:11 +0000