build(deps): bump google-github-actions/setup-gcloud from 2.1.1 to 2.1.2

Bumps [google-github-actions/setup-gcloud](https://github.com/google-github-actions/setup-gcloud) from 2.1.1 to 2.1.2. - [Release notes](https://github.com/google-github-actions/setup-gcloud/releases) - [Changelog](https://github.com/google-github-actions/setup-gcloud/blob/main/CHANGELOG.md) - [Commits](f0990588f1...6189d56e40) --- updated-dependencies: - dependency-name: google-github-actions/setup-gcloud dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>
build(deps): bump github.com/knadh/koanf/providers/file from 1.1.0 to 1.1.2 (#19 )
2024-11-11 16:35:22 +00:00 · 2024-11-11 10:46:48 -05:00 · 2024-11-11 10:46:27 -05:00 · 2024-11-11 10:46:02 -05:00 · 2024-11-11 10:45:47 -05:00 · 2024-11-11 10:45:28 -05:00
15 changed files with 581 additions and 81 deletions
--- a/.air.toml
+++ b/.air.toml
@@ -3,7 +3,7 @@ testdata_dir = "testdata"
 tmp_dir = "tmp"

 [build]
-  args_bin = ["--dev"]
+  args_bin = []
  bin = "./tmp/main"
  cmd = "go build -o ./tmp/main ."
  delay = 1000
--- a/.devcontainer/setup.sh
+++ b/.devcontainer/setup.sh
@@ -13,4 +13,5 @@ cp kindlegen/kindlegen /usr/local/bin/kindlegen
 chmod +x /usr/local/bin/kindlegen 
 rm -rf kindlegen kindlegen_linux_2.6_i386_v2_9.tar.gz

-go install github.com/air-verse/air@latest
+go install github.com/air-verse/air@latest
+go install github.com/goreleaser/goreleaser/v2@latest
--- a/.github/dependabot.yml
+++ b/.github/dependabot.yml
@@ -6,11 +6,15 @@

 version: 2
 updates:
- - package-ecosystem: "devcontainers"
-   directory: "/"
-   schedule:
-     interval: weekly
- - package-ecosystem: "github-actions"
-   directory: "/"
-   schedule:
-    interval: "weekly"
+  - package-ecosystem: "devcontainers"
+    directory: "/"
+    schedule:
+      interval: weekly
+  - package-ecosystem: "github-actions"
+    directory: "/"
+    schedule:
+      interval: "weekly"
+  - package-ecosystem: "gomod"
+    directory: "/"
+    schedule:
+      interval: "weekly"
--- a/.github/workflows/action.yml
+++ b/.github/workflows/action.yml
@@ -17,7 +17,7 @@ jobs:

    steps:
      - name: Checkout repository
-        uses: actions/checkout@v4
+        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2

      - name: Set Docker Platforms For Branch
        id: set-platforms
@@ -28,18 +28,9 @@ jobs:
              echo "DOCKER_PLATFORMS=linux/amd64" >> "$GITHUB_OUTPUT"
          fi

-      # Set up BuildKit Docker container builder to be able to build
-      # multi-platform images and export cache
-      # https://github.com/docker/setup-buildx-action
      - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@f95db51fddba0c2d1ec667646a06c2ce06100226 # v3.0.0
+        uses: docker/setup-buildx-action@c47758b77c9736f4b2ef4073d4d51994fabfe349 # v3.7.1

-      # https://github.com/docker/setup-buildx-action
-      - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v3
-
-      # Login against a Docker registry except on PR
-      # https://github.com/docker/login-action
      - name: Log into Github Container Registry
        if: github.event_name != 'pull_request'
        uses: docker/login-action@9780b0c442fbb1117ed29e0efdff1e18412f7567 # v3.3.0
@@ -55,8 +46,6 @@ jobs:
          username: ${{ secrets.DOCKERHUB_USERNAME }}
          password: ${{ secrets.DOCKERHUB_PASSWORD }}

-      # Extract metadata (tags, labels) for Docker
-      # https://github.com/docker/metadata-action
      - name: Extract Docker metadata
        id: meta
        uses: docker/metadata-action@8e5442c4ef9f78752691e2d8f8d19755c6f78e81 # v5.5.1
@@ -71,10 +60,9 @@ jobs:
            type=semver,pattern={{version}}

      # Build and push Docker image with Buildx (don't push on PR)
-      # https://github.com/docker/build-push-action
      - name: Build and push Docker image
        id: build-and-push
-        uses: docker/build-push-action@16ebe778df0e7752d2cfcbd924afdbbd89c1a755 # v6.6.1
+        uses: docker/build-push-action@4f58ea79222b3b9dc2c8bbdd6debcef730109a75 # v6.9.0
        with:
          context: .
          push: ${{ github.event_name != 'pull_request' }}
@@ -83,9 +71,13 @@ jobs:
          platforms: ${{ steps.set-platforms.outputs.DOCKER_PLATFORMS}}
          cache-from: type=gha
          cache-to: type=gha,mode=max
+          build-args: |
+            VERSION=${{ fromJSON(steps.meta.outputs.json).labels['org.opencontainers.image.version'] }}
+            REVISION=${{ fromJSON(steps.meta.outputs.json).labels['org.opencontainers.image.revision'] }}
+            BUILDTIME=${{ fromJSON(steps.meta.outputs.json).labels['org.opencontainers.image.created'] }}

      - name: Docker Hub Description
-        uses: peter-evans/dockerhub-description@v4
+        uses: peter-evans/dockerhub-description@e98e4d1628a5f3be2be7c231e50981aee98723ae # v4.0.0
        with:
          username: ${{ secrets.DOCKERHUB_USERNAME }}
          password: ${{ secrets.DOCKERHUB_PASSWORD }}
@@ -102,19 +94,47 @@ jobs:

    steps:
      - name: Checkout repository
-        uses: actions/checkout@v4
+        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
        with:
          fetch-depth: 0
        
      - name: Set up Go
-        uses: actions/setup-go@v5
+        uses: actions/setup-go@41dfa10bad2bb2ae585af6ee5bb4d7d973ad74ed # v5.1.0
        with:
          go-version: 'stable'

      - name: Run GoReleaser
-        uses: goreleaser/goreleaser-action@v6
+        uses: goreleaser/goreleaser-action@286f3b13b1b49da4ac219696163fb8c1c93e1200 # v6.0.0
        with:
          distribution: goreleaser
          version: latest
          args: release --clean
+        env:
+          GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
  
+  deploy-demo:
+    name: Deploy Demo to Cloud Run
+    runs-on: ubuntu-latest
+    environment: demo
+    needs: docker
+    permissions:
+      id-token: write
+
+    steps:
+      - name: Authenticate with Google Cloud
+        uses: google-github-actions/auth@6fc4af4b145ae7821d527454aa9bd537d1f2dc5f # v2.1.7
+        with:
+          credentials_json: ${{ secrets.GCP_SA_KEY }}
+
+      - name: Set up Cloud SDK
+        uses: google-github-actions/setup-gcloud@6189d56e4096ee891640bb02ac264be376592d6a #v2.1.2
+
+      - name: Deploy to Cloud Run
+        run: |
+          gcloud run deploy opds-proxy-demo \
+            --image=evanbuss/opds-proxy:edge \
+            --platform=managed \
+            --region=us-central1 \
+            --allow-unauthenticated
+      
+     
--- a/README.md
+++ b/README.md
@@ -17,11 +17,54 @@ By running your own OPDS Proxy you can allow eReaders to navigate and download b
    - Other: `*.epub`
 - Allows accessing HTTP basic auth OPDS feeds from primitive eReader browsers that don't natively support basic auth. 

+
 ## Getting Started
-1. Download the latest release binary or pull the latest docker image.
-2. Configure your OPDS Proxy settings via config file / environment variables.
-3. Start `opds-proxy`.
-3. Navigate your library and download books to your eReader via web interface.
+
+You can run OPDS Proxy as a Docker container or as an executable.
+
+Configuration is done via YAML file (default `config.yml`), environment variables, and command flags in that order of precedence.
+
+### Docker 
+
+Docker images are published to [Docker Hub](https://hub.docker.com/r/evanbuss/opds-proxy) and the [GitHub Container Registry](https://github.com/evan-buss/opds-proxy/pkgs/container/opds-proxy).
+
+```yaml
+services:
+  opds-proxy:
+    image: evanbuss/opds-proxy:latest
+    #image: ghcr.io/evan-buss/opds-proxy:latest
+    container_name: opds-proxy
+    # You can also use environment variables to configure the container
+    # environment:
+    #   - OPDS__PORT=5228
+    #   - OPDS__FEEDS__0__NAME=Some Feed
+    #   - OPDS__FEEDS__0__URL=http://some-feed.com/opds
+    #   - OPDS__FEEDS__0__AUTH__USERNAME=user
+    #   - OPDS__FEEDS__0__AUTH__PASSWORD=password
+    #   - OPDS__FEEDS__0__AUTH__LOCAL_ONLY=true
+    ports:
+      - 8080:8080
+    volumes:
+      - ./config.yml:/config.yml
+    restart: unless-stopped
+```
+### Executable
+
+See the [releases](https://github.com/evan-buss/opds-proxy/releases) page for the latest release.
+
+> [!NOTE]
+> The docker image includes the required dependencies to convert `.epub` files to device specific formats.
+> When running the executable, your path must include `kepubify` and `kindlegen` to enable conversion.
+> Otherwise, no conversion will be performed and the original source file will be served.
+
+```bash
+# Runs on port 8080 and looks for ./config.yml
+./opds-proxy
+
+# Runs on port 5228 and looks for ~/.config/opds-proxy-config.yml
+./opds-proxy --port 5228 --config ~/.config/opds-proxy-config.yml
+```
+

 ### Configuration Format

@@ -33,7 +76,7 @@ port: 5228
 # Optional Cookie Encryption Keys
 # If these keys aren't set, they are automatically re-generated and logged on startup.
 # When new keys are generated all existing cookies are no longer valid. 
-# You can generate new keys by running `opds-proxy -generate-keys` and then copy them to your config.
+# You can generate new keys by running `opds-proxy --generate-keys` and then copy them to your config.
 auth:
  hash_key: [32 bit hash key]
  block_key: [32 bit block key]
--- a/8
+++ b/8
@@ -23,7 +23,11 @@ RUN go mod verify

 COPY . .

-RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -ldflags "-s -w" -o opds-proxy
+ARG VERSION=dev
+ARG REVISION=unknown
+ARG BUILDTIME=unknown
+
+RUN CGO_ENABLED=0 go build -ldflags="-s -w -X main.version=${VERSION} -X main.commit=${REVISION} -X main.date=${BUILDTIME}" -o opds-proxy

 FROM gcr.io/distroless/static

@@ -31,4 +35,4 @@ COPY --from=base /usr/local/bin/kepubify /usr/local/bin/kepubify
 COPY --from=base /usr/local/bin/kindlegen /usr/local/bin/kindlegen
 COPY --from=base /src/opds-proxy/app/opds-proxy .

-CMD ["./opds-proxy"]
+ENTRYPOINT ["./opds-proxy"]
--- a/go.mod
+++ b/go.mod
@@ -4,11 +4,20 @@ go 1.22

 require (
 	github.com/knadh/koanf/parsers/yaml v0.1.0
-	github.com/knadh/koanf/providers/file v1.0.0
+	github.com/knadh/koanf/providers/file v1.1.2
 	github.com/knadh/koanf/v2 v2.1.1
 )

-require github.com/gorilla/securecookie v1.1.2
+require (
+	github.com/gorilla/securecookie v1.1.2
+	github.com/spf13/pflag v1.0.5
+)
+
+require (
+	github.com/tidwall/gjson v1.14.2 // indirect
+	github.com/tidwall/match v1.1.1 // indirect
+	github.com/tidwall/pretty v1.2.0 // indirect
+)

 require (
 	github.com/fsnotify/fsnotify v1.7.0 // indirect
@@ -16,9 +25,12 @@ require (
 	github.com/go-viper/mapstructure/v2 v2.0.0 // indirect
 	github.com/google/uuid v1.6.0
 	github.com/knadh/koanf/maps v0.1.1 // indirect
-	github.com/knadh/koanf/providers/confmap v0.1.0
+	github.com/knadh/koanf/parsers/json v0.1.0
+	github.com/knadh/koanf/providers/posflag v0.1.0
 	github.com/mitchellh/copystructure v1.2.0 // indirect
 	github.com/mitchellh/reflectwalk v1.0.2 // indirect
+	github.com/tidwall/sjson v1.2.5
+	golang.org/x/sync v0.8.0
 	golang.org/x/sys v0.22.0 // indirect
 	gopkg.in/yaml.v3 v3.0.1 // indirect
 )
--- a/go.sum
+++ b/go.sum
@@ -14,12 +14,14 @@ github.com/gorilla/securecookie v1.1.2 h1:YCIWL56dvtr73r6715mJs5ZvhtnY73hBvEF8kX
 github.com/gorilla/securecookie v1.1.2/go.mod h1:NfCASbcHqRSY+3a8tlWJwsQap2VX5pwzwo4h3eOamfo=
 github.com/knadh/koanf/maps v0.1.1 h1:G5TjmUh2D7G2YWf5SQQqSiHRJEjaicvU0KpypqB3NIs=
 github.com/knadh/koanf/maps v0.1.1/go.mod h1:npD/QZY3V6ghQDdcQzl1W4ICNVTkohC8E73eI2xW4yI=
+github.com/knadh/koanf/parsers/json v0.1.0 h1:dzSZl5pf5bBcW0Acnu20Djleto19T0CfHcvZ14NJ6fU=
+github.com/knadh/koanf/parsers/json v0.1.0/go.mod h1:ll2/MlXcZ2BfXD6YJcjVFzhG9P0TdJ207aIBKQhV2hY=
 github.com/knadh/koanf/parsers/yaml v0.1.0 h1:ZZ8/iGfRLvKSaMEECEBPM1HQslrZADk8fP1XFUxVI5w=
 github.com/knadh/koanf/parsers/yaml v0.1.0/go.mod h1:cvbUDC7AL23pImuQP0oRw/hPuccrNBS2bps8asS0CwY=
-github.com/knadh/koanf/providers/confmap v0.1.0 h1:gOkxhHkemwG4LezxxN8DMOFopOPghxRVp7JbIvdvqzU=
-github.com/knadh/koanf/providers/confmap v0.1.0/go.mod h1:2uLhxQzJnyHKfxG927awZC7+fyHFdQkd697K4MdLnIU=
-github.com/knadh/koanf/providers/file v1.0.0 h1:DtPvSQBeF+N0QLPMz0yf2bx0nFSxUcncpqQvzCxfCyk=
-github.com/knadh/koanf/providers/file v1.0.0/go.mod h1:/faSBcv2mxPVjFrXck95qeoyoZ5myJ6uxN8OOVNJJCI=
+github.com/knadh/koanf/providers/file v1.1.2 h1:aCC36YGOgV5lTtAFz2qkgtWdeQsgfxUkxDOe+2nQY3w=
+github.com/knadh/koanf/providers/file v1.1.2/go.mod h1:/faSBcv2mxPVjFrXck95qeoyoZ5myJ6uxN8OOVNJJCI=
+github.com/knadh/koanf/providers/posflag v0.1.0 h1:mKJlLrKPcAP7Ootf4pBZWJ6J+4wHYujwipe7Ie3qW6U=
+github.com/knadh/koanf/providers/posflag v0.1.0/go.mod h1:SYg03v/t8ISBNrMBRMlojH8OsKowbkXV7giIbBVgbz0=
 github.com/knadh/koanf/v2 v2.1.1 h1:/R8eXqasSTsmDCsAyYj+81Wteg8AqrV9CP6gvsTsOmM=
 github.com/knadh/koanf/v2 v2.1.1/go.mod h1:4mnTRbZCK+ALuBXHZMjDfG9y714L7TykVnZkXbMU3Es=
 github.com/mitchellh/copystructure v1.2.0 h1:vpKXTN4ewci03Vljg/q9QvCGUDttBOGBIa15WveJJGw=
@@ -28,8 +30,20 @@ github.com/mitchellh/reflectwalk v1.0.2 h1:G2LzWKi524PWgd3mLHV8Y5k7s6XUvT0Gef6zx
 github.com/mitchellh/reflectwalk v1.0.2/go.mod h1:mSTlrgnPZtwu0c4WaC2kGObEpuNDbx0jmZXqmk4esnw=
 github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
 github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
+github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
+github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
 github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk=
 github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
+github.com/tidwall/gjson v1.14.2 h1:6BBkirS0rAHjumnjHF6qgy5d2YAJ1TLIaFE2lzfOLqo=
+github.com/tidwall/gjson v1.14.2/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
+github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=
+github.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM=
+github.com/tidwall/pretty v1.2.0 h1:RWIZEg2iJ8/g6fDDYzMpobmaoGh5OLl4AXtGUGPcqCs=
+github.com/tidwall/pretty v1.2.0/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
+github.com/tidwall/sjson v1.2.5 h1:kLy8mja+1c9jlljvWTlSazM7cKDRfJuR/bOJhcY5NcY=
+github.com/tidwall/sjson v1.2.5/go.mod h1:Fvgq9kS/6ociJEDnK0Fk1cpYF4FIW6ZF7LAe+6jwd28=
+golang.org/x/sync v0.8.0 h1:3NFvSEYkUoMifnESzZl15y791HH1qU2xm6eCJU5ZPXQ=
+golang.org/x/sync v0.8.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
 golang.org/x/sys v0.22.0 h1:RI27ohtqKCnwULzJLqkv897zojh5/DwS/ENaMzUOaWI=
 golang.org/x/sys v0.22.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
--- a/html/layout.html
+++ b/html/layout.html
@@ -3,6 +3,7 @@

 <head>
  <meta charset="UTF-8" />
+  <meta name="viewport" content="width=device-width, initial-scale=0.8">
  <title>{{block "title" .}}Kobo OPDS Proxy{{end}}</title>
  <link rel="stylesheet" href="/static/style.css" />
 </head>
--- a/internal/cache/cache.go
+++ b/internal/cache/cache.go
@@ -0,0 +1,71 @@
+package cache
+
+import (
+	"sync"
+	"time"
+)
+
+type CacheEntry[T any] struct {
+	timestamp time.Time
+	Value     *T
+}
+
+type Cache[T any] struct {
+	entries map[string]*CacheEntry[T]
+	config  CacheConfig
+	mutex   sync.Mutex
+}
+
+type CacheConfig struct {
+	TTL             time.Duration
+	CleanupInterval time.Duration
+}
+
+func NewCache[T any](config CacheConfig) *Cache[T] {
+	cache := &Cache[T]{
+		entries: make(map[string]*CacheEntry[T]),
+		config:  config,
+	}
+	go cache.cleanupLoop()
+	return cache
+}
+
+func (c *Cache[T]) Set(key string, entry *T) {
+	c.mutex.Lock()
+	defer c.mutex.Unlock()
+	c.entries[key] = &CacheEntry[T]{timestamp: time.Now(), Value: entry}
+}
+
+func (c *Cache[T]) Get(key string) (*T, bool) {
+	c.mutex.Lock()
+	defer c.mutex.Unlock()
+	entry, exists := c.entries[key]
+	if !exists {
+		return nil, false
+	}
+
+	if exists && time.Since(entry.timestamp) > c.config.TTL {
+		delete(c.entries, key)
+		return nil, false
+	}
+	return entry.Value, exists
+}
+
+func (c *Cache[T]) cleanupLoop() {
+	ticker := time.NewTicker(c.config.CleanupInterval)
+	defer ticker.Stop()
+
+	for range ticker.C {
+		c.cleanEntries()
+	}
+}
+
+func (c *Cache[T]) cleanEntries() {
+	c.mutex.Lock()
+	defer c.mutex.Unlock()
+	for key, entry := range c.entries {
+		if time.Since(entry.timestamp) > c.config.TTL {
+			delete(c.entries, key)
+		}
+	}
+}
--- a/internal/debounce/debounce.go
+++ b/internal/debounce/debounce.go
@@ -0,0 +1,53 @@
+package debounce
+
+import (
+	"crypto/md5"
+	"encoding/hex"
+	"net"
+	"net/http"
+	"net/http/httptest"
+	"strconv"
+	"time"
+
+	"github.com/evan-buss/opds-proxy/internal/cache"
+	"golang.org/x/sync/singleflight"
+)
+
+func NewDebounceMiddleware(debounce time.Duration) func(next http.HandlerFunc) http.HandlerFunc {
+	responseCache := cache.NewCache[httptest.ResponseRecorder](cache.CacheConfig{CleanupInterval: time.Second, TTL: debounce})
+	singleflight := singleflight.Group{}
+
+	return func(next http.HandlerFunc) http.HandlerFunc {
+		return func(w http.ResponseWriter, r *http.Request) {
+			ip, _, _ := net.SplitHostPort(r.RemoteAddr)
+			hash := md5.Sum([]byte(ip + r.URL.Path + r.URL.RawQuery))
+			key := string(hex.EncodeToString(hash[:]))
+
+			if entry, exists := responseCache.Get(key); exists {
+				w.Header().Set("X-Debounce", "true")
+				writeResponse(entry, w)
+				return
+			}
+
+			rw, _, shared := singleflight.Do(key, func() (interface{}, error) {
+				rw := httptest.NewRecorder()
+				next(rw, r)
+				return rw, nil
+			})
+
+			recorder := rw.(*httptest.ResponseRecorder)
+			responseCache.Set(key, recorder)
+
+			w.Header().Set("X-Shared", strconv.FormatBool(shared))
+			writeResponse(recorder, w)
+		}
+	}
+}
+
+func writeResponse(rec *httptest.ResponseRecorder, w http.ResponseWriter) {
+	for k, v := range rec.Header() {
+		w.Header()[k] = v
+	}
+	w.WriteHeader(rec.Code)
+	w.Write(rec.Body.Bytes())
+}
--- a/internal/debounce/debounce_test.go
+++ b/internal/debounce/debounce_test.go
@@ -0,0 +1,106 @@
+package debounce
+
+import (
+	"net/http"
+	"net/http/httptest"
+	"sync"
+	"testing"
+	"time"
+)
+
+func TestDebounceMiddleware(t *testing.T) {
+	setup := func() (http.Handler, *int) {
+		// Mock handler that simulates a slow response
+		handlerCallCount := 0
+		mockHandler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+			handlerCallCount++
+			time.Sleep(100 * time.Millisecond) // Simulate some work
+			w.WriteHeader(http.StatusOK)
+			w.Write([]byte("OK"))
+		})
+
+		middleware := NewDebounceMiddleware(500 * time.Millisecond)
+		wrappedHandler := middleware(mockHandler)
+
+		return wrappedHandler, &handlerCallCount
+	}
+
+	t.Run("Caching Behavior", func(t *testing.T) {
+		wrappedHandler, handlerCallCount := setup()
+
+		// First request
+		req1 := httptest.NewRequest("GET", "/test", nil)
+		rec1 := httptest.NewRecorder()
+		wrappedHandler.ServeHTTP(rec1, req1)
+
+		if *handlerCallCount != 1 {
+			t.Errorf("Expected handler to be called once, got %d", handlerCallCount)
+		}
+
+		// Second request within debounce period
+		req2 := httptest.NewRequest("GET", "/test", nil)
+		rec2 := httptest.NewRecorder()
+		wrappedHandler.ServeHTTP(rec2, req2)
+
+		if *handlerCallCount != 1 {
+			t.Errorf("Expected handler to still be called once, got %d", handlerCallCount)
+		}
+
+		if rec2.Header().Get("X-Debounce") != "true" {
+			t.Error("Expected second response to be debounced")
+		}
+
+		// Wait for debounce period to expire
+		time.Sleep(600 * time.Millisecond)
+
+		// Third request after debounce period
+		req3 := httptest.NewRequest("GET", "/test", nil)
+		rec3 := httptest.NewRecorder()
+		wrappedHandler.ServeHTTP(rec3, req3)
+
+		if *handlerCallCount != 2 {
+			t.Errorf("Expected handler to be called twice, got %d", handlerCallCount)
+		}
+	})
+
+	t.Run("Singleflight Behavior", func(t *testing.T) {
+		wrappedHandler, handlerCallCount := setup()
+
+		var wg sync.WaitGroup
+		requestCount := 10
+
+		for i := 0; i < requestCount; i++ {
+			wg.Add(1)
+			go func() {
+				defer wg.Done()
+				req := httptest.NewRequest("GET", "/test", nil)
+				rec := httptest.NewRecorder()
+				wrappedHandler.ServeHTTP(rec, req)
+			}()
+		}
+
+		wg.Wait()
+
+		if *handlerCallCount != 1 {
+			t.Errorf("Expected handler to be called once for concurrent requests, got %d", handlerCallCount)
+		}
+	})
+
+	t.Run("Different Paths", func(t *testing.T) {
+		wrappedHandler, handlerCallCount := setup()
+
+		// Request to path A
+		reqA := httptest.NewRequest("GET", "/testA", nil)
+		recA := httptest.NewRecorder()
+		wrappedHandler.ServeHTTP(recA, reqA)
+
+		// Request to path B
+		reqB := httptest.NewRequest("GET", "/testB", nil)
+		recB := httptest.NewRecorder()
+		wrappedHandler.ServeHTTP(recB, reqB)
+
+		if *handlerCallCount != 2 {
+			t.Errorf("Expected handler to be called twice for different paths, got %d", handlerCallCount)
+		}
+	})
+}
--- a/internal/envextended/provider.go
+++ b/internal/envextended/provider.go
@@ -0,0 +1,105 @@
+package envextended
+
+import (
+	"errors"
+	"os"
+	"strings"
+
+	"github.com/tidwall/sjson"
+)
+
+type Env struct {
+	prefix string
+	delim  string
+	cb     func(key string, value string) (string, interface{})
+	out    string
+}
+
+func Provider(prefix, delim string, cb func(s string) string) *Env {
+	e := &Env{
+		prefix: prefix,
+		delim:  delim,
+		out:    "{}",
+	}
+	if cb != nil {
+		e.cb = func(key string, value string) (string, interface{}) {
+			return cb(key), value
+		}
+	}
+	return e
+}
+
+// ProviderWithValue works exactly the same as Provider except the callback
+// takes a (key, value) with the variable name and value and allows you
+// to modify both. This is useful for cases where you may want to return
+// other types like a string slice instead of just a string.
+func ProviderWithValue(prefix, delim string, cb func(key string, value string) (string, interface{})) *Env {
+	return &Env{
+		prefix: prefix,
+		delim:  delim,
+		cb:     cb,
+	}
+}
+
+// ReadBytes reads the contents of a file on disk and returns the bytes.
+func (e *Env) ReadBytes() ([]byte, error) {
+	// Collect the environment variable keys.
+	var keys []string
+	for _, k := range os.Environ() {
+		if e.prefix != "" {
+			if strings.HasPrefix(k, e.prefix) {
+				keys = append(keys, k)
+			}
+		} else {
+			keys = append(keys, k)
+		}
+	}
+
+	for _, k := range keys {
+		parts := strings.SplitN(k, "=", 2)
+
+		var (
+			key   string
+			value interface{}
+		)
+
+		// If there's a transformation callback,
+		// run it through every key/value.
+		if e.cb != nil {
+			key, value = e.cb(parts[0], parts[1])
+			// If the callback blanked the key, it should be omitted
+			if key == "" {
+				continue
+			}
+		} else {
+			key = parts[0]
+			value = parts[1]
+		}
+
+		if err := e.set(key, value); err != nil {
+			return []byte{}, err
+		}
+	}
+
+	if e.out == "" {
+		return []byte("{}"), nil
+	}
+
+	return []byte(e.out), nil
+}
+
+func (e *Env) set(key string, value interface{}) error {
+	out, err := sjson.Set(e.out, strings.Replace(key, e.delim, ".", -1), value)
+	if err != nil {
+		return err
+	}
+
+	e.out = out
+
+	return nil
+}
+
+// Read is not supported by the file provider.
+func (e *Env) Read() (map[string]interface{}, error) {
+	return nil, errors.New("envextended provider does not support this method")
+}
--- a/main.go
+++ b/main.go
@@ -2,24 +2,33 @@ package main

 import (
 	"encoding/hex"
-	"flag"
+	"errors"
 	"fmt"
 	"log"
 	"net/http"
 	"os"
+	"strings"

+	"github.com/evan-buss/opds-proxy/internal/envextended"
 	"github.com/gorilla/securecookie"
+	"github.com/knadh/koanf/parsers/json"
 	"github.com/knadh/koanf/parsers/yaml"
-	"github.com/knadh/koanf/providers/confmap"
 	"github.com/knadh/koanf/providers/file"
+	"github.com/knadh/koanf/providers/posflag"
 	"github.com/knadh/koanf/v2"
+	flag "github.com/spf13/pflag"
 )

+// Version information set at build time
+var version = "dev"
+var commit = "unknown"
+var date = "unknown"
+
 type ProxyConfig struct {
 	Port      string       `koanf:"port"`
 	Auth      AuthConfig   `koanf:"auth"`
 	Feeds     []FeedConfig `koanf:"feeds" `
-	isDevMode bool
+	DebugMode bool         `koanf:"debug"`
 }

 type AuthConfig struct {
@@ -40,44 +49,54 @@ type FeedConfigAuth struct {
 }

 func main() {
-	fs := flag.NewFlagSet("", flag.ContinueOnError)
-	// These aren't mapped to the config file.
-	configPath := fs.String("config", "config.yml", "config file to load")
-	generateKeys := fs.Bool("generate-keys", false, "generate cookie signing keys and exit")
-	isDevMode := fs.Bool("dev", false, "enable development mode")
+	var k = koanf.New(".")

-	port := fs.String("port", "8080", "port to listen on")
+	fs := flag.NewFlagSet("", flag.ContinueOnError)
+	fs.StringP("port", "p", "8080", "port to listen on")
+	fs.StringP("config", "c", "config.yml", "config file to load")
+	fs.Bool("generate-keys", false, "generate cookie signing keys and exit")
+	fs.BoolP("version", "v", false, "print version and exit")
+	fs.Usage = func() {
+		fmt.Println("Usage: opds-proxy [flags]")
+		fmt.Println(fs.FlagUsages())
+		os.Exit(0)
+	}
 	if err := fs.Parse(os.Args[1:]); err != nil {
-		log.Fatal(err)
+		log.Fatalf("error parsing flags: %v", err)
 	}

-	if *generateKeys {
+	if showVersion, _ := fs.GetBool("version"); showVersion {
+		fmt.Println("opds-proxy")
+		fmt.Printf("  Version: %s\n", version)
+		fmt.Printf("  Commit: %s\n", commit)
+		fmt.Printf("  Build Date: %s\n", date)
+		os.Exit(0)
+	}
+
+	if generate, _ := fs.GetBool("generate-keys"); generate {
 		displayKeys()
 		os.Exit(0)
 	}

-	var k = koanf.New(".")
-
-	// Load config file from disk.
-	// Feed options must be defined here.
-	if err := k.Load(file.Provider(*configPath), yaml.Parser()); err != nil && !os.IsNotExist(err) {
-		log.Fatal(err)
+	// YAML Config
+	configPath, _ := fs.GetString("config")
+	if err := k.Load(file.Provider(configPath), yaml.Parser()); err != nil && !os.IsNotExist(err) {
+		log.Fatalf("error loading config file: %v", err)
 	}

-	// Selectively add command line options to the config. Overriding the config file.
-	if err := k.Load(confmap.Provider(map[string]interface{}{
-		"port": *port,
-	}, "."), nil); err != nil {
-		log.Fatal(err)
+	// Environment Variables Config
+	if err := k.Load(envextended.ProviderWithValue("OPDS", ".", envCallback), json.Parser()); err != nil {
+		log.Fatalf("error loading environment variables: %v", err)
+	}
+
+	// CLI Flags Config
+	if err := k.Load(posflag.Provider(fs, ".", k), nil); err != nil {
+		log.Fatalf("error loading CLI flags: %v", err)
 	}

 	config := ProxyConfig{}
 	k.Unmarshal("", &config)

-	if len(config.Feeds) == 0 {
-		log.Fatal("No feeds defined in config")
-	}
-
 	if config.Auth.HashKey == "" || config.Auth.BlockKey == "" {
 		log.Println("Generating new cookie signing credentials")
 		hashKey, blockKey := displayKeys()
@@ -86,16 +105,17 @@ func main() {
 		config.Auth.BlockKey = blockKey
 	}

-	// This should only be set by the command line flag,
-	// so we don't use koanf to set this.
-	config.isDevMode = *isDevMode
+	if err := config.Validate(); err != nil {
+		log.Fatalf("invalid configuration: %v", err)
+	}
+
 	server, err := NewServer(&config)
 	if err != nil {
-		log.Fatal(err)
+		log.Fatalf("error creating server: %v", err)
 	}

 	if err = server.Serve(); err != nil && err != http.ErrServerClosed {
-		log.Fatal(err)
+		log.Fatalf("error serving: %v", err)
 	}
 }

@@ -110,3 +130,36 @@ func displayKeys() (string, string) {

 	return hashKey, blockKey
 }
+
+func envCallback(key string, value string) (string, interface{}) {
+	key = strings.TrimPrefix(key, "OPDS__")
+	key = strings.ReplaceAll(key, "__", ".")
+	key = strings.ToLower(key)
+	return key, value
+}
+
+func (c *ProxyConfig) Validate() error {
+	if c.Port == "" {
+		return errors.New("port is required")
+	}
+
+	if c.Auth.HashKey == "" || c.Auth.BlockKey == "" {
+		return errors.New("auth.hash_key and auth.block_key are required")
+	}
+
+	if len(c.Feeds) == 0 {
+		return errors.New("at least one feed must be defined")
+	}
+
+	for _, feed := range c.Feeds {
+		if feed.Name == "" {
+			return errors.New("feed.name is required")
+		}
+
+		if feed.Url == "" {
+			return errors.New("feed.url is required")
+		}
+	}
+
+	return nil
+}
--- a/server.go
+++ b/server.go
@@ -19,6 +19,7 @@ import (

 	"github.com/evan-buss/opds-proxy/convert"
 	"github.com/evan-buss/opds-proxy/html"
+	"github.com/evan-buss/opds-proxy/internal/debounce"
 	"github.com/evan-buss/opds-proxy/opds"
 	"github.com/google/uuid"
 	"github.com/gorilla/securecookie"
@@ -66,15 +67,21 @@ func NewServer(config *ProxyConfig) (*Server, error) {
 		return nil, err
 	}

-	if !config.isDevMode {
+	if !config.DebugMode {
 		slog.SetDefault(slog.New(slog.NewJSONHandler(os.Stdout, nil)))
 	}

 	s := securecookie.New(hashKey, blockKey)

+	// Kobo issues 2 requests for each clicked link. This middleware ensures
+	// we only process the first request and provide the same response for the second.
+	// This becomes more important when the requests aren't idempotent, such as triggering
+	// a download.
+	debounceMiddleware := debounce.NewDebounceMiddleware(time.Millisecond * 100)
+
 	router := http.NewServeMux()
 	router.Handle("GET /{$}", requestMiddleware(handleHome(config.Feeds)))
-	router.Handle("GET /feed", requestMiddleware(handleFeed("tmp/", config.Feeds, s)))
+	router.Handle("GET /feed", requestMiddleware(debounceMiddleware(handleFeed("tmp/", config.Feeds, s))))
 	router.Handle("/auth", requestMiddleware(handleAuth(s)))
 	router.Handle("GET /static/", http.FileServer(http.FS(html.StaticFiles())))

@@ -120,7 +127,12 @@ func requestMiddleware(next http.Handler) http.Handler {
 		r = r.WithContext(ctx)

 		next.ServeHTTP(w, r)
-		log.Info("Request Completed", slog.String("duration", time.Since(start).String()))
+
+		log.Info("Request Completed",
+			slog.String("duration", time.Since(start).String()),
+			slog.Bool("debounce", w.Header().Get("X-Debounce") == "true"),
+			slog.Bool("shared", w.Header().Get("X-Shared") == "true"),
+		)
 	})
 }

@@ -208,6 +220,7 @@ func handleFeed(outputDir string, feeds []FeedConfig, s *securecookie.SecureCook
 				handleError(r, w, "Failed to render feed", err)
 				return
 			}
+			return
 		}

 		mutex.Lock()
@@ -332,6 +345,10 @@ func getCredentials(r *http.Request, feeds []FeedConfig, s *securecookie.SecureC
 			continue
 		}

+		if feed.Auth == nil || feed.Auth.Username == "" || feed.Auth.Password == "" {
+			continue
+		}
+
 		// Only set feed credentials for local requests
 		// when the auth config has local_only flag
 		isLocal := r.Context().Value(isLocalRequest).(bool)
@@ -339,10 +356,6 @@ func getCredentials(r *http.Request, feeds []FeedConfig, s *securecookie.SecureC
 			continue
 		}

-		if feed.Auth == nil || feed.Auth.Username == "" || feed.Auth.Password == "" {
-			continue
-		}
-
 		return &Credentials{Username: feed.Auth.Username, Password: feed.Auth.Password}
 	}
Author	SHA1	Message	Date
dependabot[bot]	e6451d73b4	build(deps): bump google-github-actions/setup-gcloud from 2.1.1 to 2.1.2 Bumps [google-github-actions/setup-gcloud](https://github.com/google-github-actions/setup-gcloud) from 2.1.1 to 2.1.2. - [Release notes](https://github.com/google-github-actions/setup-gcloud/releases) - [Changelog](https://github.com/google-github-actions/setup-gcloud/blob/main/CHANGELOG.md) - [Commits](`f0990588f1...6189d56e40`) --- updated-dependencies: - dependency-name: google-github-actions/setup-gcloud dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com>	2024-11-11 16:35:22 +00:00
dependabot[bot]	baab651482	build(deps): bump github.com/knadh/koanf/providers/file from 1.1.0 to 1.1.2 (#19 ) Bumps [github.com/knadh/koanf/providers/file](https://github.com/knadh/koanf) from 1.1.0 to 1.1.2. - [Release notes](https://github.com/knadh/koanf/releases) - [Commits](https://github.com/knadh/koanf/compare/v1.1.0...providers/file/v1.1.2) --- updated-dependencies: - dependency-name: github.com/knadh/koanf/providers/file dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-11-11 10:46:48 -05:00
dependabot[bot]	c2a29b5cb3	build(deps): bump actions/setup-go from 5.0.2 to 5.1.0 (#21 ) Bumps [actions/setup-go](https://github.com/actions/setup-go) from 5.0.2 to 5.1.0. - [Release notes](https://github.com/actions/setup-go/releases) - [Commits](`0a12ed9d6a...41dfa10bad`) --- updated-dependencies: - dependency-name: actions/setup-go dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-11-11 10:46:27 -05:00
dependabot[bot]	9cb0fa32b5	build(deps): bump docker/build-push-action from 6.7.0 to 6.9.0 (#15 ) Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 6.7.0 to 6.9.0. - [Release notes](https://github.com/docker/build-push-action/releases) - [Commits](`5cd11c3a4c...4f58ea7922`) --- updated-dependencies: - dependency-name: docker/build-push-action dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-11-11 10:46:02 -05:00
dependabot[bot]	a02d0b582f	build(deps): bump docker/setup-buildx-action from 3.6.1 to 3.7.1 (#17 ) Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 3.6.1 to 3.7.1. - [Release notes](https://github.com/docker/setup-buildx-action/releases) - [Commits](`988b5a0280...c47758b77c`) --- updated-dependencies: - dependency-name: docker/setup-buildx-action dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-11-11 10:45:47 -05:00
dependabot[bot]	8e6a5d2234	build(deps): bump actions/checkout from 4.1.7 to 4.2.2 (#22 ) Bumps [actions/checkout](https://github.com/actions/checkout) from 4.1.7 to 4.2.2. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](`692973e3d9...11bd71901b`) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-11-11 10:45:28 -05:00
dependabot[bot]	3612201c77	build(deps): bump google-github-actions/auth from 2.1.5 to 2.1.7 (#23 ) Bumps [google-github-actions/auth](https://github.com/google-github-actions/auth) from 2.1.5 to 2.1.7. - [Release notes](https://github.com/google-github-actions/auth/releases) - [Changelog](https://github.com/google-github-actions/auth/blob/main/CHANGELOG.md) - [Commits](`62cf5bd3e4...6fc4af4b14`) --- updated-dependencies: - dependency-name: google-github-actions/auth dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-11-11 10:45:09 -05:00
dependabot[bot]	05203649d8	build(deps): bump google-github-actions/auth from 2.1.4 to 2.1.5 (#14 ) Bumps [google-github-actions/auth](https://github.com/google-github-actions/auth) from 2.1.4 to 2.1.5. - [Release notes](https://github.com/google-github-actions/auth/releases) - [Changelog](https://github.com/google-github-actions/auth/blob/main/CHANGELOG.md) - [Commits](`f112390a2d...62cf5bd3e4`) --- updated-dependencies: - dependency-name: google-github-actions/auth dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-08-31 12:22:02 -04:00
dependabot[bot]	3d01b5eac5	build(deps): bump docker/build-push-action from 6.6.1 to 6.7.0 (#13 ) Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 6.6.1 to 6.7.0. - [Release notes](https://github.com/docker/build-push-action/releases) - [Commits](`16ebe778df...5cd11c3a4c`) --- updated-dependencies: - dependency-name: docker/build-push-action dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-08-22 17:23:51 -04:00
Evan Buss	0f0540549d	feat: request deduplication / debouncing Kobo eReaders have a buggy browser that makes 2 requests for the same HTTP resource when you click a link. This change ensures that requests within a certain time frame from the same IP, for the same path / query params will only be executed a single time. We record the http request response and replay it for the second request. If we get 2 simultaneous requests, we use the sync/singleflight library to ensure only the first request is actually processed. The second waits for the shared result of the first. This probably adds latency since some requests are blocked while we determine if we already have a cache entry, but for a simple service like this I don't think it matters.	2024-08-18 18:37:42 +00:00
dependabot[bot]	5d45afd419	build(deps): bump github.com/knadh/koanf/providers/file from 1.0.0 to 1.1.0 (#11 ) Bumps [github.com/knadh/koanf/providers/file](https://github.com/knadh/koanf) from 1.0.0 to 1.1.0. - [Release notes](https://github.com/knadh/koanf/releases) - [Commits](https://github.com/knadh/koanf/compare/v1.0.0...v1.1.0) --- updated-dependencies: - dependency-name: github.com/knadh/koanf/providers/file dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-08-12 10:02:07 -04:00
dependabot[bot]	129aa984cd	build(deps): bump docker/setup-buildx-action from 3.0.0 to 3.6.1 (#10 ) Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 3.0.0 to 3.6.1. - [Release notes](https://github.com/docker/setup-buildx-action/releases) - [Commits](`f95db51fdd...988b5a0280`) --- updated-dependencies: - dependency-name: docker/setup-buildx-action dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>	2024-08-12 10:01:51 -04:00
Evan Buss	3e3b662aa9	fix: mobile device scale Add `viewport` meta tag so site scales properly on mobile devices	2024-08-12 13:57:50 +00:00
Evan Buss	105a72c931	ci: use correct cloud run name	2024-08-11 19:42:35 -04:00
Evan Buss	03664b3f58	build: automatic deploy to cloud run	2024-08-11 23:20:56 +00:00
Evan Buss	33ab199b21	fix: panic when auth not specified	2024-08-11 20:20:41 +00:00
Evan Buss	7e80ae1718	ci: fix build-args	2024-08-11 15:33:03 -04:00
Evan Buss	54687383d1	ci: fix ldflags	2024-08-11 19:20:48 +00:00
Evan Buss	9da7ea1bbc	feat: environment configuration Environment variables can now be used to configure any config property including the feeds list. This makes it easier to use in environments without access to config files like GCR. Some may prefer not to have a separate config file as well. Also added build metadata to the docker image and binaries.	2024-08-11 18:47:46 +00:00
Evan Buss	d8d35b6cef	ci: pin action versions to specific commit	2024-08-11 11:16:07 -04:00
Evan Buss	1ad71172f1	docs: update getting started	2024-08-11 05:25:20 +00:00
Evan Buss	c4a5f54a62	ci: goreleaser missing github token	2024-08-11 00:46:59 -04:00