Thanks @masonasons for the tip that yt-dlp can download from RSS feeds. Here's the command I came up with to download all available podcast items from a feed in chronological order (oldest first) to nicely numbered files with the title and date.

yt-dlp --no-abort-on-error --color "no_color" --download-archive ".download_history" --windows-filenames --embed-metadata --embed-chapters --playlist-items "::-1" --output "%(n_entries+1-playlist_index)02d %(title)s (%(upload_date>%B %d %Y)s).%(ext)s" --format bestaudio "example.com/rss"