profile
viewpoint

issue commentopenzim/youtube

WebP polyfill error on welcome poster

This issue has been automatically marked as stale because it has not had recent activity. It will be now be reviewed manually. Thank you for your contributions.

rgaudin

comment created time in 20 hours

push eventopenzim/youtube

renaud gaudin

commit sha 038428afe4a0420de190d1d8a5d3819281b48bd1

released v2.1.13

view details

renaud gaudin

commit sha 84959a462c557f9b3be9dcd3ac9d8f4e06c77ee1

back to dev

view details

push time in 2 days

created tagopenzim/youtube

tagv2.1.13

Create a ZIM file from a Youtube channel/username/playlist

created time in 2 days

push eventopenzim/youtube

renaud gaudin

commit sha 68ed8e0be720e257be6de86c2683f00d03234c21

using docker publish v5

view details

push time in 2 days

push eventopenzim/youtube

renaud gaudin

commit sha 3a9fa0fcb7dede1bddfb9d4d688b36cd47b863e0

Added zh-Hant-TW mapping

view details

push time in 2 days

issue commentopenzim/youtube

Not a valid iso language name/code

As for teded, it was zh-Hant-TW.

kelson42

comment created time in 2 days

issue commentopenzim/youtube

Not a valid iso language name/code

And you mentioned khan-videos_es_playlists but this one has noting to do with this. it's a recipe issue.

kelson42

comment created time in 2 days

issue commentopenzim/youtube

Not a valid iso language name/code

@kelson42, youtube sometimes uses non-standard codes for subtitles. That's something we have to react to when we encounter it but it's not a regression.

I've added mapping for the khan-videos-bn culprit (zh-Hans-CN) and I've tried to find the list of all possible codes from the YT subtitle uploads UI. I found a couple of codes that would have failed and mapped those as well.

Unfortunately, that UI had no reference of that zh-Hans-CN code for instance so it means the UI doesn't expose all the codes they are using, meaning we might encounter a similar issue in the future.

I've added a debug print of such code so that in the event of a similar issue, we'll save a lot of time and be able to fix it immediately.

So, next time, open a new ticket with the lang code.

kelson42

comment created time in 2 days

push eventopenzim/youtube

renaud gaudin

commit sha 4e08952947f698fb6d2288165fadfdf1efad6aba

Fixed #121: extended YT lang code mapping Added lang code mapping for `zh-Hans-CN`, `mo` and `sh`. Added debug print of incorrect language code in case of failure

view details

push time in 2 days

issue closedopenzim/youtube

Not a valid iso language name/code

https://farm.openzim.org/pipeline/5f5f3154cf078db7bac5284e/debug

closed time in 2 days

kelson42

issue closedopenzim/youtube

New crash scenario pattern with mindfield scraping

See https://farm.openzim.org/pipeline/b2e47e70159a9be6d08f6ff5/debug

[youtube2zim::2021-01-10 13:20:01,636] INFO:download all author's profile pictures [youtube2zim::2021-01-10 13:20:01,636] DEBUG:query youtube-api for Channel #UC6nSFpj9HTCZ5t-N3Rm3-HA [youtube2zim::2021-01-10 13:20:01,832] INFO:update general metadata [youtube2zim::2021-01-10 13:20:01,832] DEBUG:query youtube-api for Channel #CSANnvayMtMLizmxt8koEHH5nj1ZR7xq6Pnu-VMYg [youtube2zim::2021-01-10 13:20:01,959] ERROR:FAILED. An error occurred: 'items' [youtube2zim::2021-01-10 13:20:01,959] ERROR:'items' Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/youtube2zim-2.1.12-py3.8.egg/youtube2zim/entrypoint.py", line 202, in main scraper.run() File "/usr/local/lib/python3.8/site-packages/youtube2zim-2.1.12-py3.8.egg/youtube2zim/scraper.py", line 310, in run self.update_metadata() File "/usr/local/lib/python3.8/site-packages/youtube2zim-2.1.12-py3.8.egg/youtube2zim/scraper.py", line 727, in update_metadata main_channel_json = get_channel_json(self.main_channel_id) File "/usr/local/lib/python3.8/site-packages/youtube2zim-2.1.12-py3.8.egg/youtube2zim/youtube.py", line 88, in get_channel_json channel_json = req.json()["items"][0] KeyError: 'items'

closed time in 2 days

kelson42

issue commentopenzim/youtube

New crash scenario pattern with mindfield scraping

This is happening because of inconsistent data returned by the Youtube API:

When querying playlist PLZRRxQcaEjA4qyEuYfAMCazlL0vQDkIj2, the API tells us the creator_id is Channel CSANnvayMtMLizmxt8koEHH5nj1ZR7xq6Pnu-VMYg.

As you can see, this channel doesn't raise a 404 as incorrect IDs does but the page is blank. If you click on the playlist link above, it says the channel is Channel vsauce1 (which apparently is same as User Vsauce.

So we have no way, from the API, to know about Vsauce and we can only query for that channel ID returned which failes because there is no item returned by the API for it.

I don't want to fix this because:

  • this shouldn't happen. this is clearly a bug on YT API.
  • making it work will have unwanted side effects.
  • this is happening only on this recipe and is about past playlists that won't evolve. Actually, it was supposed to be publicly available only for a month and that's why the recipe was created in the first place.

I guess this is due to the fact that this was a Youtube Originals content (reserved to paying members) that got opened later.

I suggest we close this and disable the recipe.

kelson42

comment created time in 2 days

pull request commentopenzim/openedx

Fixes #160 - Fail properly on downloads

This pull request has been automatically marked as stale because it has not had recent activity. It will be now be reviewed manually. Thank you for your contributions.

satyamtg

comment created time in 2 days

issue commentopenzim/youtube

Not a valid iso language name/code

Same problem with :

  • https://farm.openzim.org/recipes/khan-videos_es_playlists
  • https://farm.openzim.org/recipes/khan-videos_bn_playlists
kelson42

comment created time in 2 days

issue commentopenzim/gutenberg

Move to OPDS catalog (Simplify Gutenberg scraping)

we'll probably go straight to opds 2.0 (json-based) sometime this year

kelson42

comment created time in 3 days

issue commentopenzim/gutenberg

Move to OPDS catalog (Simplify Gutenberg scraping)

@eshellman From our perspective having a proper OPDS would be preferable, because this is a standard.

kelson42

comment created time in 4 days

IssuesEvent

issue commentopenzim/sotoki

Add different sort order active/oldest/votes using javascript

I'm again unsure after the helpful comment of @dattaz. If I understand right now, the goal would be to add the following buttons? image

I'm not really sure "Active" makes a lot of sense, but would the two other ones be doable in pure javascript (not the buttons, the sorting) based on the DOM infos?

amirouche

comment created time in 5 days

pull request commentopenzim/gutenberg

Now JS will be added during setup

@hksingh4545 Thx

hksingh4545

comment created time in 5 days

PR opened openzim/gutenberg

Now JS will be added during setup

Now JS of DataTables-1.10.13 and Responsive-2.1.1 will be added during setup

+45 -17451

0 comment

6 changed files

pr created time in 5 days

issue commentopenzim/sotoki

Add different sort order active/oldest/votes using javascript

Let's close it.

amirouche

comment created time in 5 days

startedArkanosis/microstatus

started time in 6 days

issue closedopenzim/youtube

UnboundLocalError: local variable 'meta' referenced before assignment

[download] Destination: /output/tmpobb8pd_4/videos/Ouy2ocDbFYs/video.f251.webm

[download] 0.0% of 19.67MiB at 405.05KiB/s ETA 00:49 [download] 0.0% of 19.67MiB at 1.16MiB/s ETA 00:17 [download] 0.0% of 19.67MiB at 2.67MiB/s ETA 00:07 [download] 0.1% of 19.67MiB at 5.64MiB/s ETA 00:03 [download] 0.2% of 19.67MiB at 2.10MiB/s ETA 00:09 [download] 0.3% of 19.67MiB at 2.71MiB/s ETA 00:07 [download] 0.6% of 19.67MiB at 3.60MiB/s ETA 00:05 [download] 1.3% of 19.67MiB at 5.44MiB/s ETA 00:03 [download] 2.5% of 19.67MiB at 8.77MiB/s ETA 00:02 [download] 5.1% of 19.67MiB at 14.54MiB/s ETA 00:01 [download] 10.2% of 19.67MiB at 24.86MiB/s ETA 00:00 [download] 20.3% of 19.67MiB at 25.95MiB/s ETA 00:00 [download] 40.7% of 19.67MiB at 30.64MiB/s ETA 00:00 [download] 49.6% of 19.67MiB at 28.05MiB/s ETA 00:00 [download] 49.6% of 19.67MiB at 416.76KiB/s ETA 00:24 [download] 49.6% of 19.67MiB at 1.19MiB/s ETA 00:08 [download] 49.6% of 19.67MiB at 2.73MiB/s ETA 00:03 [download] 49.7% of 19.67MiB at 5.78MiB/s ETA 00:01 [download] 49.8% of 19.67MiB at 2.10MiB/s ETA 00:04 [download] 49.9% of 19.67MiB at 2.59MiB/s ETA 00:03 [download] 50.2% of 19.67MiB at 3.58MiB/s ETA 00:02 [download] 50.9% of 19.67MiB at 5.24MiB/s ETA 00:01 [download] 52.1% of 19.67MiB at 8.46MiB/s ETA 00:01 [download] 54.7% of 19.67MiB at 14.15MiB/s ETA 00:00 [download] 59.8% of 19.67MiB at 24.26MiB/s ETA 00:00 [download] 69.9% of 19.67MiB at 30.94MiB/s ETA 00:00 [download] 90.3% of 19.67MiB at 35.36MiB/s ETA 00:00 [download] 98.8% of 19.67MiB at 35.40MiB/s ETA 00:00 [download] 98.8% of 19.67MiB at 438.87KiB/s ETA 00:00 [download] 98.8% of 19.67MiB at 1.25MiB/s ETA 00:00 [download] 98.9% of 19.67MiB at 2.87MiB/s ETA 00:00 [download] 98.9% of 19.67MiB at 6.03MiB/s ETA 00:00 [download] 99.0% of 19.67MiB at 2.09MiB/s ETA 00:00 [download] 99.1% of 19.67MiB at 2.67MiB/s ETA 00:00 [download] 99.5% of 19.67MiB at 3.56MiB/s ETA 00:00 [download] 100.0% of 19.67MiB at 5.10MiB/s ETA 00:00 [download] 100% of 19.67MiB in 00:00 [ffmpeg] Merging formats into "/output/tmpobb8pd_4/videos/Ouy2ocDbFYs/video.webm" Deleting original file /output/tmpobb8pd_4/videos/Ouy2ocDbFYs/video.f244.webm (pass -k to keep) Deleting original file /output/tmpobb8pd_4/videos/Ouy2ocDbFYs/video.f251.webm (pass -k to keep) [zimscraperlib::2021-01-11 02:18:34,602] INFO:Encode /output/tmpobb8pd_4/videos/Ouy2ocDbFYs/video.webm -> /output/tmpobb8pd_4/videos/Ouy2ocDbFYs/video.webm video format = .webm [youtube2zim::2021-01-11 02:31:44,534] DEBUG:Uploading video file for Ouy2ocDbFYs to cache ... [youtube2zim::2021-01-11 02:31:49,370] ERROR:webm/low/Ouy2ocDbFYs failed to upload to cache: Failed to upload /output/tmpobb8pd_4/videos/Ouy2ocDbFYs/video.webm to org-kiwix-youtube/webm/low/Ouy2ocDbFYs: An error occurred (TemporarilyUnavailable) when calling the CreateMultipartUpload operation (reached max retries: 4): Maximum number of server active requests exceeded [youtube2zim::2021-01-11 02:31:49,370] DEBUG:Attempting to download thumbnail for Ouy2ocDbFYs from cache... [youtube2zim::2021-01-11 02:31:58,227] ERROR:FAILED. An error occurred: local variable 'meta' referenced before assignment [youtube2zim::2021-01-11 02:31:58,227] ERROR:local variable 'meta' referenced before assignment Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/youtube2zim-2.1.12-py3.8.egg/youtube2zim/entrypoint.py", line 202, in main scraper.run() File "/usr/local/lib/python3.8/site-packages/youtube2zim-2.1.12-py3.8.egg/youtube2zim/scraper.py", line 294, in run succeeded, failed = self.download_video_files( File "/usr/local/lib/python3.8/site-packages/youtube2zim-2.1.12-py3.8.egg/youtube2zim/scraper.py", line 506, in download_video_files return self.download_video_files_batch(options, self.videos_ids) File "/usr/local/lib/python3.8/site-packages/youtube2zim-2.1.12-py3.8.egg/youtube2zim/scraper.py", line 698, in download_video_files_batch if self.download_video(video_id, options) and self.download_thumbnail( File "/usr/local/lib/python3.8/site-packages/youtube2zim-2.1.12-py3.8.egg/youtube2zim/scraper.py", line 651, in download_thumbnail if self.download_from_cache(s3_key, thumbnail_path, preset.VERSION): File "/usr/local/lib/python3.8/site-packages/youtube2zim-2.1.12-py3.8.egg/youtube2zim/scraper.py", line 564, in download_from_cache if not self.s3_storage.has_object_matching_meta( File "/usr/local/lib/python3.8/site-packages/kiwixstorage-0.4-py3.8.egg/kiwixstorage/init.py", line 430, in has_object_matching_meta return meta.get(tag) == value UnboundLocalError: local variable 'meta' referenced before assignment

In c’est pas sorcier recipe https://farm.openzim.org/pipeline/0cb0223bedd943b9b7537ff5/debug

closed time in 6 days

kelson42

issue commentopenzim/youtube

UnboundLocalError: local variable 'meta' referenced before assignment

upstream bug https://github.com/kiwix/python_storagelib/issues/5

kelson42

comment created time in 6 days

issue closedopenzim/youtube

Unable to scrape keylearning

https://farm.openzim.org/pipeline/5e291209e3a713c9cbdebff5/debug

[youtube2zim::2021-01-11 06:20:15,354] ERROR:FAILED. An error occurred: 404 Client Error: Not Found for url: https://www.googleapis.com/youtube/v3/playlistItems?playlistId=+PLrei6UENqoSpmwTS-ALRpGR3icX6HlRmB&part=snippet%2CcontentDetails&key=AIzaSyCjggfpmMSXFLoMhZQgyMlPBHoUXgQb5uc&maxResults=50
[youtube2zim::2021-01-11 06:20:15,354] ERROR:404 Client Error: Not Found for url: https://www.googleapis.com/youtube/v3/playlistItems?playlistId=+PLrei6UENqoSpmwTS-ALRpGR3icX6HlRmB&part=snippet%2CcontentDetails&key=AIzaSyCjggfpmMSXFLoMhZQgyMlPBHoUXgQb5uc&maxResults=50
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/youtube2zim-2.1.12-py3.8.egg/youtube2zim/entrypoint.py", line 202, in main
    scraper.run()
  File "/usr/local/lib/python3.8/site-packages/youtube2zim-2.1.12-py3.8.egg/youtube2zim/scraper.py", line 274, in run
    self.extract_videos_list()
  File "/usr/local/lib/python3.8/site-packages/youtube2zim-2.1.12-py3.8.egg/youtube2zim/scraper.py", line 458, in extract_videos_list
    videos_json = get_videos_json(playlist.playlist_id)
  File "/usr/local/lib/python3.8/site-packages/youtube2zim-2.1.12-py3.8.egg/youtube2zim/youtube.py", line 185, in get_videos_json
    req.raise_for_status()
  File "/usr/local/lib/python3.8/site-packages/requests/models.py", line 943, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://www.googleapis.com/youtube/v3/playlistItems?playlistId=+PLrei6UENqoSpmwTS-ALRpGR3icX6HlRmB&part=snippet%2CcontentDetails&key=AIzaSyCjggfpmMSXFLoMhZQgyMlPBHoUXgQb5uc&maxResults=50

closed time in 6 days

kelson42

issue commentopenzim/youtube

Unable to scrape keylearning

Spaces in playlist IDs. Recipe mistake.

kelson42

comment created time in 6 days

issue openedopenzim/youtube

Unable to scrape keylearning

https://farm.openzim.org/pipeline/5e291209e3a713c9cbdebff5/debug

[youtube2zim::2021-01-11 06:20:15,354] ERROR:FAILED. An error occurred: 404 Client Error: Not Found for url: https://www.googleapis.com/youtube/v3/playlistItems?playlistId=+PLrei6UENqoSpmwTS-ALRpGR3icX6HlRmB&part=snippet%2CcontentDetails&key=AIzaSyCjggfpmMSXFLoMhZQgyMlPBHoUXgQb5uc&maxResults=50
[youtube2zim::2021-01-11 06:20:15,354] ERROR:404 Client Error: Not Found for url: https://www.googleapis.com/youtube/v3/playlistItems?playlistId=+PLrei6UENqoSpmwTS-ALRpGR3icX6HlRmB&part=snippet%2CcontentDetails&key=AIzaSyCjggfpmMSXFLoMhZQgyMlPBHoUXgQb5uc&maxResults=50
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/youtube2zim-2.1.12-py3.8.egg/youtube2zim/entrypoint.py", line 202, in main
    scraper.run()
  File "/usr/local/lib/python3.8/site-packages/youtube2zim-2.1.12-py3.8.egg/youtube2zim/scraper.py", line 274, in run
    self.extract_videos_list()
  File "/usr/local/lib/python3.8/site-packages/youtube2zim-2.1.12-py3.8.egg/youtube2zim/scraper.py", line 458, in extract_videos_list
    videos_json = get_videos_json(playlist.playlist_id)
  File "/usr/local/lib/python3.8/site-packages/youtube2zim-2.1.12-py3.8.egg/youtube2zim/youtube.py", line 185, in get_videos_json
    req.raise_for_status()
  File "/usr/local/lib/python3.8/site-packages/requests/models.py", line 943, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://www.googleapis.com/youtube/v3/playlistItems?playlistId=+PLrei6UENqoSpmwTS-ALRpGR3icX6HlRmB&part=snippet%2CcontentDetails&key=AIzaSyCjggfpmMSXFLoMhZQgyMlPBHoUXgQb5uc&maxResults=50

created time in 6 days

issue openedopenzim/youtube

UnboundLocalError: local variable 'meta' referenced before assignment

[download] Destination: /output/tmpobb8pd_4/videos/Ouy2ocDbFYs/video.f251.webm

[download] 0.0% of 19.67MiB at 405.05KiB/s ETA 00:49 [download] 0.0% of 19.67MiB at 1.16MiB/s ETA 00:17 [download] 0.0% of 19.67MiB at 2.67MiB/s ETA 00:07 [download] 0.1% of 19.67MiB at 5.64MiB/s ETA 00:03 [download] 0.2% of 19.67MiB at 2.10MiB/s ETA 00:09 [download] 0.3% of 19.67MiB at 2.71MiB/s ETA 00:07 [download] 0.6% of 19.67MiB at 3.60MiB/s ETA 00:05 [download] 1.3% of 19.67MiB at 5.44MiB/s ETA 00:03 [download] 2.5% of 19.67MiB at 8.77MiB/s ETA 00:02 [download] 5.1% of 19.67MiB at 14.54MiB/s ETA 00:01 [download] 10.2% of 19.67MiB at 24.86MiB/s ETA 00:00 [download] 20.3% of 19.67MiB at 25.95MiB/s ETA 00:00 [download] 40.7% of 19.67MiB at 30.64MiB/s ETA 00:00 [download] 49.6% of 19.67MiB at 28.05MiB/s ETA 00:00 [download] 49.6% of 19.67MiB at 416.76KiB/s ETA 00:24 [download] 49.6% of 19.67MiB at 1.19MiB/s ETA 00:08 [download] 49.6% of 19.67MiB at 2.73MiB/s ETA 00:03 [download] 49.7% of 19.67MiB at 5.78MiB/s ETA 00:01 [download] 49.8% of 19.67MiB at 2.10MiB/s ETA 00:04 [download] 49.9% of 19.67MiB at 2.59MiB/s ETA 00:03 [download] 50.2% of 19.67MiB at 3.58MiB/s ETA 00:02 [download] 50.9% of 19.67MiB at 5.24MiB/s ETA 00:01 [download] 52.1% of 19.67MiB at 8.46MiB/s ETA 00:01 [download] 54.7% of 19.67MiB at 14.15MiB/s ETA 00:00 [download] 59.8% of 19.67MiB at 24.26MiB/s ETA 00:00 [download] 69.9% of 19.67MiB at 30.94MiB/s ETA 00:00 [download] 90.3% of 19.67MiB at 35.36MiB/s ETA 00:00 [download] 98.8% of 19.67MiB at 35.40MiB/s ETA 00:00 [download] 98.8% of 19.67MiB at 438.87KiB/s ETA 00:00 [download] 98.8% of 19.67MiB at 1.25MiB/s ETA 00:00 [download] 98.9% of 19.67MiB at 2.87MiB/s ETA 00:00 [download] 98.9% of 19.67MiB at 6.03MiB/s ETA 00:00 [download] 99.0% of 19.67MiB at 2.09MiB/s ETA 00:00 [download] 99.1% of 19.67MiB at 2.67MiB/s ETA 00:00 [download] 99.5% of 19.67MiB at 3.56MiB/s ETA 00:00 [download] 100.0% of 19.67MiB at 5.10MiB/s ETA 00:00 [download] 100% of 19.67MiB in 00:00 [ffmpeg] Merging formats into "/output/tmpobb8pd_4/videos/Ouy2ocDbFYs/video.webm" Deleting original file /output/tmpobb8pd_4/videos/Ouy2ocDbFYs/video.f244.webm (pass -k to keep) Deleting original file /output/tmpobb8pd_4/videos/Ouy2ocDbFYs/video.f251.webm (pass -k to keep) [zimscraperlib::2021-01-11 02:18:34,602] INFO:Encode /output/tmpobb8pd_4/videos/Ouy2ocDbFYs/video.webm -> /output/tmpobb8pd_4/videos/Ouy2ocDbFYs/video.webm video format = .webm [youtube2zim::2021-01-11 02:31:44,534] DEBUG:Uploading video file for Ouy2ocDbFYs to cache ... [youtube2zim::2021-01-11 02:31:49,370] ERROR:webm/low/Ouy2ocDbFYs failed to upload to cache: Failed to upload /output/tmpobb8pd_4/videos/Ouy2ocDbFYs/video.webm to org-kiwix-youtube/webm/low/Ouy2ocDbFYs: An error occurred (TemporarilyUnavailable) when calling the CreateMultipartUpload operation (reached max retries: 4): Maximum number of server active requests exceeded [youtube2zim::2021-01-11 02:31:49,370] DEBUG:Attempting to download thumbnail for Ouy2ocDbFYs from cache... [youtube2zim::2021-01-11 02:31:58,227] ERROR:FAILED. An error occurred: local variable 'meta' referenced before assignment [youtube2zim::2021-01-11 02:31:58,227] ERROR:local variable 'meta' referenced before assignment Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/youtube2zim-2.1.12-py3.8.egg/youtube2zim/entrypoint.py", line 202, in main scraper.run() File "/usr/local/lib/python3.8/site-packages/youtube2zim-2.1.12-py3.8.egg/youtube2zim/scraper.py", line 294, in run succeeded, failed = self.download_video_files( File "/usr/local/lib/python3.8/site-packages/youtube2zim-2.1.12-py3.8.egg/youtube2zim/scraper.py", line 506, in download_video_files return self.download_video_files_batch(options, self.videos_ids) File "/usr/local/lib/python3.8/site-packages/youtube2zim-2.1.12-py3.8.egg/youtube2zim/scraper.py", line 698, in download_video_files_batch if self.download_video(video_id, options) and self.download_thumbnail( File "/usr/local/lib/python3.8/site-packages/youtube2zim-2.1.12-py3.8.egg/youtube2zim/scraper.py", line 651, in download_thumbnail if self.download_from_cache(s3_key, thumbnail_path, preset.VERSION): File "/usr/local/lib/python3.8/site-packages/youtube2zim-2.1.12-py3.8.egg/youtube2zim/scraper.py", line 564, in download_from_cache if not self.s3_storage.has_object_matching_meta( File "/usr/local/lib/python3.8/site-packages/kiwixstorage-0.4-py3.8.egg/kiwixstorage/init.py", line 430, in has_object_matching_meta return meta.get(tag) == value UnboundLocalError: local variable 'meta' referenced before assignment

In c’est pas sorcier recipe https://farm.openzim.org/pipeline/0cb0223bedd943b9b7537ff5/debug

created time in 6 days

issue openedopenzim/youtube

New crash scenario pattern with mindfield scraping

See https://farm.openzim.org/pipeline/b2e47e70159a9be6d08f6ff5/debug

[youtube2zim::2021-01-10 13:20:01,636] INFO:download all author's profile pictures [youtube2zim::2021-01-10 13:20:01,636] DEBUG:query youtube-api for Channel #UC6nSFpj9HTCZ5t-N3Rm3-HA [youtube2zim::2021-01-10 13:20:01,832] INFO:update general metadata [youtube2zim::2021-01-10 13:20:01,832] DEBUG:query youtube-api for Channel #CSANnvayMtMLizmxt8koEHH5nj1ZR7xq6Pnu-VMYg [youtube2zim::2021-01-10 13:20:01,959] ERROR:FAILED. An error occurred: 'items' [youtube2zim::2021-01-10 13:20:01,959] ERROR:'items' Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/youtube2zim-2.1.12-py3.8.egg/youtube2zim/entrypoint.py", line 202, in main scraper.run() File "/usr/local/lib/python3.8/site-packages/youtube2zim-2.1.12-py3.8.egg/youtube2zim/scraper.py", line 310, in run self.update_metadata() File "/usr/local/lib/python3.8/site-packages/youtube2zim-2.1.12-py3.8.egg/youtube2zim/scraper.py", line 727, in update_metadata main_channel_json = get_channel_json(self.main_channel_id) File "/usr/local/lib/python3.8/site-packages/youtube2zim-2.1.12-py3.8.egg/youtube2zim/youtube.py", line 88, in get_channel_json channel_json = req.json()["items"][0] KeyError: 'items'

created time in 7 days

more