profile
viewpoint
Hervé Beraud 4383 Red Hat - Openstack Core Developer France https://herve.beraud.io @openstack release management PTL & core developer - senior software engineer at Red Hat. FOSS contributor. curl -L tiny.cc/5zvo1y 2>/dev/null | sh

4383/barcode 5

Simple lightweight python barcode EAN generator. Output format SVG and ASCII

4383/bookmarks 1

Common bookmarks and useful links

4383/4383.github.io 0

Hervé Beraud Personal Website

4383/anishot 0

Animate a long screenshot

4383/ansible 0

Ansible is a radically simple IT automation platform that makes your applications and systems easier to deploy. Avoid writing scripts or custom code to deploy and update your applications— automate in a language that approaches plain English, using SSH, with no agents to install on remote systems.

4383/ansible-swarm 0

hands on lab dedicated to ansible and swarm

4383/apkbuild 0

docker container for building apk packages for alpine linux distribution

4383/aports 0

Mirror of aports repository

4383/asciidoc-book-toolchain 0

Book publishing toolchain based on AsciiDoc

fork h-vetinari/allennlp-models-feedstock

A conda-smithy repository for allennlp-models.

fork in 14 minutes

fork h-vetinari/allennlp-optuna-feedstock

A conda-smithy repository for allennlp-optuna.

fork in 21 minutes

issue commentsqlalchemy/dogpile.cache

Make the Region Invalidation Strategy more flexible

I like the extensions that @zzzeek proposed, however...

Is this best implemented with a region invalidation though?

This looks like a usecase for “wraps” or a custom deserializer/backend. At least I have done similar things with that concept. (I’m not sure if the internal payload is available in wraps hooks or not). You can just perform the date operations there, and issue a miss so it repopulates with the generator.

JonathanWylie

comment created time in an hour

startedmicrosoft/MT-DNN

started time in 2 hours

startedmicrosoft/DeBERTa

started time in 2 hours

startedkevinduh/san_mrc

started time in 2 hours

startedLiyuanLucasLiu/RAdam

started time in 2 hours

startednamisan/mt-dnn

started time in 2 hours

issue commentsqlalchemy/dogpile.cache

Make the Region Invalidation Strategy more flexible

looks like #38 , sort of midway through, is where this came about. I'll name some of the usual suspects in case this issue is interesting to them, @morganfainberg @jvanasco

JonathanWylie

comment created time in 2 hours

issue commentsqlalchemy/dogpile.cache

Make the Region Invalidation Strategy more flexible

hi there -

I agree it seems strange I went through all the trouble to add a custom region invalidation strategy and did not think that the key, which is available, should be passed through as well. that extension was created for someone else's use case, I have no idea what it was.

here is a proposed API. if you could vet this for sanity etc. that would be very helpful, I rarely work with dogpile.cache so I came up with this through rote:

diff --git a/dogpile/cache/region.py b/dogpile/cache/region.py
index 65ed334..224b148 100644
--- a/dogpile/cache/region.py
+++ b/dogpile/cache/region.py
@@ -104,6 +104,9 @@ class RegionInvalidationStrategy:
                 return (self._hard_invalidated and
                         timestamp < self._hard_invalidated)
 
+            def key_is_hard_invalidated(self, key, timestamp):
+                return self.is_hard_invalidated(timestamp)
+
             def was_soft_invalidated(self):
                 return bool(self._soft_invalidated)
 
@@ -111,6 +114,9 @@ class RegionInvalidationStrategy:
                 return (self._soft_invalidated and
                         timestamp < self._soft_invalidated)
 
+            def key_is_soft_invalidated(self, key, timestamp):
+                return self.is_soft_invalidated(timestamp)
+
     The custom implementation is injected into a :class:`.CacheRegion`
     at configure time using the
     :paramref:`.CacheRegion.configure.region_invalidator` parameter::
@@ -164,6 +170,21 @@ class RegionInvalidationStrategy:
 
         raise NotImplementedError()
 
+    def key_is_hard_invalidated(self, key: KeyType, timestamp: float) -> bool:
+        """Check timestamp and key to determine if it was hard invalidated.
+
+        Calls :meth:`.RegionInvalidator.is_hard_invalidated` by default.
+
+        :return: Boolean. True if ``timestamp`` is older than
+         the last region invalidation time and region is invalidated
+         in soft mode.
+
+        .. versionadded:: 1.1.2
+
+        """
+
+        return self.is_hard_invalidated(timestamp)
+
     def is_soft_invalidated(self, timestamp: float) -> bool:
         """Check timestamp to determine if it was soft invalidated.
 
@@ -175,6 +196,21 @@ class RegionInvalidationStrategy:
 
         raise NotImplementedError()
 
+    def key_is_soft_invalidated(self, key: KeyType, timestamp: float) -> bool:
+        """Check timestamp and key to determine if it was soft invalidated.
+
+        Calls :meth:`.RegionInvalidator.is_soft_invalidated` by default.
+
+        :return: Boolean. True if ``timestamp`` is older than
+         the last region invalidation time and region is invalidated
+         in soft mode.
+
+        .. versionadded:: 1.1.2
+
+        """
+
+        return self.is_soft_invalidated(timestamp)
+
     def is_invalidated(self, timestamp: float) -> bool:
         """Check timestamp to determine if it was invalidated.
 
@@ -185,6 +221,18 @@ class RegionInvalidationStrategy:
 
         raise NotImplementedError()
 
+    def key_is_invalidated(self, key: KeyType, timestamp: float) -> bool:
+        """Check timestamp and key to determine if it was invalidated.
+
+        :return: Boolean. True if ``timestamp`` is older than
+         the last region invalidation time.
+
+        .. versionadded:: 1.1.2
+
+        """
+
+        return self.is_invalidated(timestamp)
+
     def was_soft_invalidated(self) -> bool:
         """Indicate the region was invalidated in soft mode.
 
@@ -761,21 +809,21 @@ class CacheRegion:
             key = self.key_mangler(key)
         value = self._get_from_backend(key)
         value = self._unexpired_value_fn(expiration_time, ignore_expiration)(
-            value
+            key, value
         )
 
         return value.payload
 
     def _unexpired_value_fn(self, expiration_time, ignore_expiration):
         if ignore_expiration:
-            return lambda value: value
+            return lambda key, value: value
         else:
             if expiration_time is None:
                 expiration_time = self.expiration_time
 
             current_time = time.time()
 
-            def value_fn(value):
+            def value_fn(key, value):
                 if value is NO_VALUE:
                     return value
                 elif (
@@ -783,8 +831,8 @@ class CacheRegion:
                     and current_time - value.metadata["ct"] > expiration_time
                 ):
                     return NO_VALUE
-                elif self.region_invalidator.is_invalidated(
-                    value.metadata["ct"]
+                elif self.region_invalidator.key_is_invalidated(
+                    key, value.metadata["ct"]
                 ):
                     return NO_VALUE
                 else:
@@ -838,7 +886,7 @@ class CacheRegion:
         return [
             value.payload if value is not NO_VALUE else value
             for value in (
-                _unexpired_value_fn(value) for value in backend_values
+                _unexpired_value_fn(key, value) for key, value in zip(keys, backend_values)
             )
         ]
 
@@ -858,7 +906,7 @@ class CacheRegion:
             log.debug("No value present for key: %r", orig_key)
         elif value.metadata["v"] != value_version:
             log.debug("Dogpile version update for key: %r", orig_key)
-        elif self.region_invalidator.is_hard_invalidated(value.metadata["ct"]):
+        elif self.region_invalidator.key_is_hard_invalidated(orig_key, value.metadata["ct"]):
             log.debug("Hard invalidation detected for key: %r", orig_key)
         else:
             return False
@@ -965,7 +1013,7 @@ class CacheRegion:
                 raise NeedRegenerationException()
 
             ct = cast(CachedValue, value).metadata["ct"]
-            if self.region_invalidator.is_soft_invalidated(ct):
+            if self.region_invalidator.key_is_soft_invalidated(key, ct):
                 if expiration_time is None:
                     raise exception.DogpileCacheException(
                         "Non-None expiration time required "

JonathanWylie

comment created time in 2 hours

startedmdbooth/gerrit-recheck

started time in 3 hours

fork tirkarthi/aimrecords

AimRecords — Record-oriented data storage

fork in 4 hours

startedaimhubio/aimrecords

started time in 4 hours

fork strategist922/tinyfecVPN

A VPN Designed for Lossy Links, with Build-in Forward Error Correction(FEC) Support. Improves your Network Quality on a High-latency Lossy Link.

fork in 4 hours

startedwangyu-/tinyfecVPN

started time in 4 hours

fork strategist922/UDPspeeder

A Tunnel which Improves your Network Quality on a High-latency Lossy Link by using Forward Error Correction, possible for All Traffics(TCP/UDP/ICMP)

fork in 4 hours

startedwangyu-/UDPspeeder

started time in 4 hours

startedwseaton/rustyaml

started time in 4 hours

fork Kristinita/pandoc-url2cite

Effortlessly and transparently add correctly styled citations to your markdown paper given only a URL

fork in 5 hours

startedgarmeeh/next-seo

started time in 5 hours

fork strategist922/ncovis-2020

covid-19 舆论和新闻的可视化平台,获得了中国计算机学会、阿里云和机器之心等举办的疫情可视化比赛铜奖。🔥

https://pearmini.gitee.io/ncovis-2020/

fork in 6 hours

startedpearmini/ncovis-2020

started time in 6 hours

fork strategist922/ZHIHU-crawler3.1

同样是feed流的方式获得信息,这个版本已经有了一定的对抗能力,还是要尊重爬虫时的访问速度,给出足够的时间间隔,不能占用太多请求资源

fork in 6 hours

startedSparrow-zhu/ZHIHU-crawler3.1

started time in 6 hours

fork mscherer/asciidoctor

:gem: A fast, open source text processor and publishing toolchain, written in Ruby, for converting AsciiDoc content to HTML 5, DocBook 5, and other formats.

https://asciidoctor.org

fork in 6 hours

fork haraldh/sgx

Library for Intel SGX

fork in 6 hours

fork strategist922/Zhihu-Spider-1

知乎全站爬虫(用户、话题、提问、回答、评论)以及一些工具,例如知乎搜索,热榜。。。

fork in 6 hours

startedHanZhuo-ii/Zhihu-Spider

started time in 6 hours

fork strategist922/zhihu-3

项目没有维护了, fork 吧

fork in 7 hours

startedshanelau/zhihu

started time in 7 hours

more