Keep collecting podcast data when a feed was removed from PodcastIndex#413
Merged
andygrunwald merged 1 commit intoJul 19, 2026
mainEngineeringKiosk/GermanTechPodcasts:mainfrom
andygrunwald/tolerate-removed-podcastindex-feedsEngineeringKiosk/GermanTechPodcasts:andygrunwald/tolerate-removed-podcastindex-feedsCopy head branch name to clipboard
Merged
Keep collecting podcast data when a feed was removed from PodcastIndex#413andygrunwald merged 1 commit intomainEngineeringKiosk/GermanTechPodcasts:mainfrom andygrunwald/tolerate-removed-podcastindex-feedsEngineeringKiosk/GermanTechPodcasts:andygrunwald/tolerate-removed-podcastindex-feedsCopy head branch name to clipboard
andygrunwald merged 1 commit into
mainEngineeringKiosk/GermanTechPodcasts:mainfrom
andygrunwald/tolerate-removed-podcastindex-feedsEngineeringKiosk/GermanTechPodcasts:andygrunwald/tolerate-removed-podcastindex-feedsCopy head branch name to clipboard
Conversation
The scheduled "Podcast data" workflow aborted while processing
it-at-db.json (feed id 685245):
json: cannot unmarshal array into Go struct field Podcast.feed
of type podcastindex.PodcastFeed:
{"status":"true","query":{"id":"685245"},"feed":[],
"description":"No feeds match this id."}
Once a feed is removed from the PodcastIndex, the API answers with an
empty JSON array as "feed" instead of a feed object. Podcast.Feed is
typed as a struct, so decoding failed and the error aborted the run.
A single dead podcast therefore stopped the remaining ~50 of 112 files
from being refreshed.
Decode "feed" through a custom UnmarshalJSON that treats an array as
"no feed found" and expose that via Podcast.Found. The collector skips
such a podcast and keeps the data collected in the past, following the
same tolerance we already apply to failing cover image downloads.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
andygrunwald
deleted the
andygrunwald/tolerate-removed-podcastindex-feeds
branch
July 19, 2026 08:24
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The scheduled
Podcast dataworkflow fails. Run 29679381516 aborted while processinggenerated/it-at-db.json(feed id 685245):Once a feed is removed from the PodcastIndex, the API answers with an empty JSON array as
feedinstead of a feed object.Podcast.Feedis typed as a struct, so decoding fails, andcmdCollectPodcastDatareturns that error immediately. A single delisted podcast therefore stopped the remaining ~50 of 112 files from being refreshed.IT@DB has been
archive: truewith no new episode since Aug 2025, so it having disappeared from the index is expected — it just shouldn't take the whole run down with it.Change
app/podcastindex/podcasts.go: decodefeedthrough a customUnmarshalJSONthat treats a leading[as "no feed found", and expose the outcome via a newPodcast.Foundfield.app/cmd/collectPodcastData.go: when!p.Found, log it andcontinue. Image download, episode retrieval and the write-back are skipped, so the existing generated JSON stays byte-identical. This follows the tolerance we already apply when a cover image download fails but an older image is on disk.Every other error path in the loop keeps its current fail-fast behaviour.
Verification
go build ./...,go vet ./...andgo test ./...pass. Addedapp/podcastindex/podcasts_test.go, which decodes the exact failing payload from the run log (expects no error,Found == false) and a normal feed response (expects the fields populated). These are the first test files in the repo.Not verified locally: the end-to-end run against the live API, which needs the real key/secret. Worth watching the next scheduled run — it should finish, log
PodcastIndex has no feed for id 685245 anymore, and leavegenerated/it-at-db.jsonunchanged.Noticed, not fixed
collectPodcastData.go:150-162: when an image download fails but an old file exists,DoesImageExistsOnDisk(path, true)returns an absolute path while the sibling branch returns a bare filename. The followingfilepath.Join(imageFolder, podcastImageName)then writes a brokenimages//home/runner/.../images/wp-sofa.pngintopodcastInfo.Image. That branch was hit in this same run (wp-sofa, HTTP 423). Unrelated to this crash, left alone.🤖 Generated with Claude Code