profile
viewpoint
Matti Schneider MattiSG Nomadic, mostly EU https://mattischneider.fr Transdisciplinary engineer.

MattiSG/adblock 149

Block ads and trackers system-wide, before a request is even issued.

ambanum/pariscall.international 5

Public website for the Paris Call for Trust and Security in Cyberspace / Site public de l'Appel de Paris pour la confiance et la sécurité dans le cyberespace

MattiSG/Dash-MooTools 5

DEPRECATED MooTools documentation generator for Dash

ambanum/DormantTwitterAccountsDetector 2

[HACKATHON] Detect "dormant" bots on Twitter

MattiSG/decider-mes-impots 2

Une application web pour orienter le type d'actions que je souhaite que mes impôts financent.

MattiSG/deep-sleep 2

Let your (old) Mac go to hibernation (deep sleep)

MattiSG/arc2 1

ARC RDF Classes for PHP

MattiSG/comment-voter 1

Le guide le plus simple du vote en France.

AgileFrance/site-conference-2016 0

Archive du site de la conférence Agile France 2016

pull request commentambanum/CGUs

Remove obsolete code and dependencies

This was moved to https://github.com/ambanum/TOSBack-CGUs-bridge 🙂

Ndpnt

comment created time in 5 hours

pull request commentambanum/CGUs

Document common practice for choosing a service name and service ID

Why was this closed? 🤔

Ndpnt

comment created time in 5 hours

PR opened ambanum/CGUs

Document discussions around document types

Make #88 and #89 discoverable to explain jurisprudence around document types.

Fixes #89.

+2 -0

0 comment

1 changed file

pr created time in 8 hours

create barnchambanum/CGUs

branch : document-service-types

created branch time in 8 hours

issue closedambanum/CGUs

Tasks stay pending in tosback-import

Follow-up from #71. I think some tasks hit their 10 seconds timeout but then still keep running in the background. Script run ends with this and then hangs for a long time:

Next task (1 tasks left, running 5 in parallel)
Sologig Privacy Policy start
Sologig Privacy Policy skip
Saving services/Washingtonpost.json
Washingtonpost Privacy Policy done
Pending: [
  'Zimbio - Privacy Policy - http://www.livingly.com/privacy-policy/',
  'Urbanoutfitters - Privacy Policy - http://www.urbanoutfitters.com/urban/help/privacy_security.jsp',
  'Zynga - Privacy Policy - http://company.zynga.com/about/privacy-center/privacy-policy#sharing-information'
]
Could not fetch http://www.urbanoutfitters.com/urban/help/privacy_security.jsp
Urbanoutfitters Privacy Policy fail
Pending: [
  'Zimbio - Privacy Policy - http://www.livingly.com/privacy-policy/',
  'Zynga - Privacy Policy - http://company.zynga.com/about/privacy-center/privacy-policy#sharing-information'
]
Could not filter http://www.livingly.com/privacy-policy/
Zimbio Privacy Policy fail
Pending: [
  'Zynga - Privacy Policy - http://company.zynga.com/about/privacy-center/privacy-policy#sharing-information'
]
Could not fetch http://company.zynga.com/about/privacy-center/privacy-policy#sharing-information
Zynga Privacy Policy fail
Pending: []
Could not fetch http://tracking.quisma.com/policy.htm
Could not fetch http://www.providecommerce.com/privacy.aspx
Could not fetch http://www.seattletimescompany.com/notices/notice2.html
Could not filter http://www.washingtonpost.com/privacy-policy/2011/11/18/gIQASIiaiN_print.html

closed time in 9 hours

michielbdejong

issue commentambanum/CGUs

Tasks stay pending in tosback-import

This should be fixed in #102 via #139 🙂

michielbdejong

comment created time in 9 hours

issue closedambanum/CGUs

Use task queue

As described in https://github.com/ambanum/CGUs/pull/71#issuecomment-674956397

closed time in 9 hours

michielbdejong

issue commentambanum/CGUs

Use task queue

This has been implemented in #139 🙂

michielbdejong

comment created time in 9 hours

issue closedambanum/CGUs

Only notify for a subset of services

Hi,

I would be interested in getting notifications only for a subset of services (especially with the upcoming inclusion of the tosback declarations).

I could handle it on my own, running a separate copy of CGUs, but would it this feature be interesting more broadly? Seems notification code is in https://github.com/ambanum/CGUs/blob/d979a3ac789e071933b8c2940a24b2a4e1bca8de/src/notifier/index.js but I am not very familiar with your SendInBlue setup for notifications.

Depending on your feedback, I can help writing patch / code on this :)

Best,

closed time in 9 hours

LucasVerneyDGE

issue commentambanum/CGUs

Only notify for a subset of services

As discussed in https://github.com/ambanum/CGUs/pull/131#issuecomment-696736211, we are now using GitHub-generated Atom feeds for commits. This has been documented in #150 and #154, and fixes this issue since RSS can easily be turned into email generators with third-party tools.

As documented in https://github.com/ambanum/CGUs/pull/131#issuecomment-696831472, we don't have the intention to fully depend on GitHub over time, but it is too early in the life of the product to invest in an independent solution. We first want to assess use cases before generating our own RSS feeds and possibly offer dedicated email notifications.

Thanks again @LucasVerneyDGE for opening this discussion, for contributing a PoC that can act as a basis for future implementations, and for reviewing the documentation! Please let us know of any feedback from users that would use the RSS feature 😃

LucasVerneyDGE

comment created time in 9 hours

issue commentambanum/CGUs

Harmonize whitespaces

In the given example, there would have been a notification in any case, since the last line changed.

Let's see if there are other cases where space style results in noise :)

LucasVerneyDGE

comment created time in 9 hours

PR opened ambanum/CGUs

Improve RSS feeds documentation
+52 -45

0 comment

2 changed files

pr created time in 9 hours

create barnchambanum/CGUs

branch : document-rss-feeds

created branch time in 9 hours

Pull request review commentambanum/CGUs

Document RSS feeds

 Cliquez sur une modification pour voir en quoi elle consiste (par exemple [celle  Vous pouvez vous abonner pour recevoir un courriel lorsqu'un document est modifié en [remplissant ce formulaire](https://59692a77.sibforms.com/serve/MUIEAKuTv3y67e27PkjAiw7UkHCn0qVrcD188cQb-ofHVBGpvdUWQ6EraZ5AIb6vJqz3L8LDvYhEzPb2SE6eGWP35zXrpwEFVJCpGuER9DKPBUrifKScpF_ENMqwE_OiOZ3FdCV2ra-TXQNxB2sTEL13Zj8HU7U0vbbeF7TnbFiW8gGbcOa5liqmMvw_rghnEB2htMQRCk6A3eyj). -**Attention, ce service est en version bêta et vous risquez de recevoir de nombreuses notifications !** Vous pourrez vous désabonner en répondant à n'importe quel courriel reçu.+**Attention, ce service est en version bêta et vous risquez de recevoir de nombreuses notifications !** Vous pourrez vous désabonner en répondant à n'importe quel courriel reçu. +Il est également possible d'être notifié pour un service ou un document spécifique en s'abonnant à un des flux RSS disponibles.++_Un flux RSS est une ressource accessible en ligne qui contient des informations sur les derniers contenus publiés par un site web comme par exemple leur date de publication et l'adresse pour les consulter sur le site.+Lorsque cette ressource est mise à jour, votre lecteur de flux vous prévient automatiquement vous pouvez aller voir la mise à jour._++### Flux RSS disponibles++| Est mis à jour pour | URL |+|--|--|+| l'ensemble des services et documents | `https://github.com/ambanum/CGUs-versions/commits.atom` |+|l'ensemble des documents d'un service| `https://github.com/ambanum/CGUs-versions/commits/master/$serviceId.atom.` Remplacer __$serviceId__ par l'identifiant du service. |+| un document spécifique d'un service | `https://github.com/ambanum/CGUs-versions/commits/master/$serviceId/$documentType.md.atom` Remplacer __$serviceId__ par l'idantifiant du service et __$documentType__ par le type du document.<br> Par exemple pour le document `Privacy Policy` de `Google`, l'adresse est `https://github.com/ambanum/CGUs-versions/commits/master/Google/Privacy%20Policy.md.atom` |

Ça dépend du lecteur de flux. Avec Leaf par exemple, l'espace n'est pas échappé. Je vais le réintroduire.

Ndpnt

comment created time in 9 hours

PullRequestReviewEvent

push eventambanum/CGUs

Matti Schneider

commit sha c37ea5c4130bfe2b9b8f73edc96640c26bbd7cf0

Improve test name

view details

push time in 9 hours

Pull request review commentambanum/CGUs

Improve robustness by using queues

 describe('CGUs', () => {         after(resetGitRepository);          it('does not refilter the service on error', async () => {
        it('emits an inaccessibleContent event when an error happens during refiltering', async () => {
Ndpnt

comment created time in 10 hours

PullRequestReviewEvent
PullRequestReviewEvent
PullRequestReviewEvent
PullRequestReviewEvent

push eventambanum/CGUs

Matti Schneider

commit sha 99de617526ec223461a3d3c1c1fd0d64e5abbbc6

Improve validate script robustness

view details

push time in 10 hours

Pull request review commentambanum/CGUs

Fix services validation on CI

     "posttest": "npm run lint",     "test": "NODE_ENV=test mocha --file './test/helper.js' --recursive \"./src/**/*.test.js\"",     "test:debug": "NODE_ENV=test mocha --file './test/helper.js' --recursive \"./src/**/*.test.js\" --inspect-brk --exit",-    "validate": "NODE_ENV=validation mocha --delay --file './test/helper.js' scripts/validation/validate.js",+    "validate": "NODE_ENV=validation node --unhandled-rejections=strict ./node_modules/mocha/bin/mocha --delay --file './test/helper.js' scripts/validation/validate.js",

Using the bin alias is more robust than navigating the filesystem within the module.

    "validate": "NODE_ENV=validation node --unhandled-rejections=strict ./node_modules/.bin/mocha --delay --file './test/helper.js' scripts/validation/validate.js",
Ndpnt

comment created time in 10 hours

PullRequestReviewEvent
PullRequestReviewEvent
GollumEvent
GollumEvent

Pull request review commentambanum/CGUs

Improve robustness by using queues

 import { InaccessibleContentError } from './errors.js';  const __dirname = path.dirname(new URL(import.meta.url).pathname); const SERVICE_DECLARATIONS_PATH = path.resolve(__dirname, '../../', config.get('serviceDeclarationsPath'));+const NUMBER_OF_TRACK_DOCUMENT_WORKERS = 20;+const NUMBER_OF_REFILTER_WORKERS = 1;

Is there any reason for not having a higher parallelization maximum? If no, it would be great to accelerate that step. If yes, it seems critical to me to document it.

Ndpnt

comment created time in 5 days

Pull request review commentambanum/CGUs

Improve robustness by using queues

 export default class CGUs extends events.EventEmitter {   }    async _forEachDocumentOf(servicesIds = [], callback) {-    const documentPromises = servicesIds.flatMap(serviceId => {+    servicesIds.forEach(serviceId => {       const { documents } = this._serviceDeclarations[serviceId];-      return Object.keys(documents).map(async type => {-        try {-          await callback({-            serviceId,-            document: {-              type,-              ...documents[type]-            }-          });-        } catch (error) {-          if (error instanceof InaccessibleContentError) {-            return this.emit('inaccessibleContent', serviceId, type, error);+      Object.keys(documents).forEach(type => {+        callback({+          serviceId,+          document: {+            type,+            ...documents[type]           }-          throw error;

Why do we remove this event emission from the _forEachDocumentOf interface?

Ndpnt

comment created time in 5 days

Pull request review commentambanum/CGUs

Improve robustness by using queues

 export default class CGUs extends events.EventEmitter {    async init() {     if (!this._serviceDeclarations) {+      this.trackDocumentChangesQueue = async.queue(async ({ serviceDocument }) => this.trackDocumentChanges(serviceDocument), NUMBER_OF_TRACK_DOCUMENT_WORKERS);+      this.refilterDocumentsQueue = async.queue(async ({ serviceDocument }) => this.refilterAndRecordDocument(serviceDocument), NUMBER_OF_REFILTER_WORKERS);++      const queueErrorHandler = (error, { serviceDocument }) => {+        const { serviceId, document: { type } } = serviceDocument;+        if (error instanceof InaccessibleContentError) {+          return this.emit('inaccessibleContent', error, serviceId, type);+        }+        this.emit('error', error, serviceId, type);+        this.trackDocumentChangesQueue.kill();+        this.refilterDocumentsQueue.kill();+      };

Declare a method rather than an anonymous function in init?

Ndpnt

comment created time in 5 days

Pull request review commentambanum/CGUs

Improve robustness by using queues

 export default class Recorder {    async commit(filePath, message) {     try {-      if (!await this.git.hasChanges(filePath)) {-        return;-      }-

Why can this be safely removed?

Ndpnt

comment created time in 5 days

Pull request review commentambanum/CGUs

Improve robustness by using queues

 export default class CGUs extends events.EventEmitter {    async init() {     if (!this._serviceDeclarations) {+      this.trackDocumentChangesQueue = async.queue(async ({ serviceDocument }) => this.trackDocumentChanges(serviceDocument), NUMBER_OF_TRACK_DOCUMENT_WORKERS);
      this.trackDocumentChangesQueue = async.queue(async ({ document }) => this.trackDocumentChanges(document), NUMBER_OF_TRACK_DOCUMENT_WORKERS);
Ndpnt

comment created time in 5 days

Pull request review commentambanum/CGUs

Improve robustness by using queues

 import { InaccessibleContentError } from './errors.js';  const __dirname = path.dirname(new URL(import.meta.url).pathname); const SERVICE_DECLARATIONS_PATH = path.resolve(__dirname, '../../', config.get('serviceDeclarationsPath'));+const NUMBER_OF_TRACK_DOCUMENT_WORKERS = 20;+const NUMBER_OF_REFILTER_WORKERS = 1;
const MAX_PARALLEL_DOCUMENTS_TRACKS = 20;
const MAX_PARALLEL_REFILTERS = 1;
Ndpnt

comment created time in 5 days

PullRequestReviewEvent
PullRequestReviewEvent

pull request commentambanum/CGUs

Fix services validation on CI

Ok, ask me again for a review when fixed :wink:

Ndpnt

comment created time in 5 days

Pull request review commentambanum/CGUs

Fix services validation on CI

 jobs:      steps:     - uses: actions/checkout@v2+      with:+        fetch-depth: 0+        ref: ${{ github.event.pull_request.head.sha }}+    - run: git checkout HEAD+    - run: git fetch origin master:master

I don't understand why we have to do this manually. Let's 📞

Ndpnt

comment created time in 5 days

PullRequestReviewEvent
PullRequestReviewEvent

pull request commentambanum/CGUs

Fix services validation on CI

Fails because Error: Could not find any service with id "./test/helper.js", which is due to the fragile arguments.slice(6) call. See https://github.com/ambanum/CGUs/pull/110#discussion_r484869875 for more context.

Ndpnt

comment created time in 5 days

GollumEvent

pull request commentambanum/CGUs

Do not track images from Google Privacy / ToS

🎉 88% file size reduction, went down from 240 to 31 kB and decreased chances of noise, thanks @LucasVerneyDGE! 😃

LucasVerneyDGE

comment created time in 6 days

Pull request review commentambanum/CGUs

Track Facebook Data Transfer Addendum

         "cleanUrls"       ],       "select": "#content"+    },+    "Data Controller Agreement": {

I would go with Data Processor Agreement, since this addendum applies to the Commercial Terms (hence targeting business entities, not individuals), and applies “to the extent that Facebook Ireland Limited is acting as your Processor of EU Data under the applicable product terms”, establishes that “by providing EU Data to Facebook Ireland Limited as your Processor”, the commercial entity that provides user data agrees to keep subprocessor information confidential and a couple other things that only relate to subprocessing.

I would consider https://www.facebook.com/legal/terms/dataprocessing as the Data Controller Agreement, as it defines how user data should be handled jointly by both Facebook and the commercial entity that transfers user data to Facebook.

I'll ask for advice from people who know more 🙂

LucasVerneyDGE

comment created time in 6 days

PullRequestReviewEvent
PullRequestReviewEvent

issue openedambanum/CGUs

[Facebook] [Data Transfer] Track the Data Transfer Addendum

In https://github.com/ambanum/CGUs-versions/commit/49832cdfa, Facebook removes a lot of content from its Commercial Terms towards a new “Data Transfer Addendum” that should be tracked.

created time in 7 days

pull request commentambanum/CGUs

First PoC of a RSS notifier

It also work on a subfolder history.

OMG that's awesome 😱

we definitively need a clear documentation to allow user to easily subscribe to document updates

Yes please, let's add that to the user's manual.

a generated list of all available URLs feeds

Let's just document how to get the RSS URL from the service name. When we want to be more user-friendly we'll have to create a full UI anyway 😉 I'm not sure of the added value of a list of RSS feeds at the moment.

My only concern with the Github approach is to have a too tight dependency on Github

I agree! But I'm way more in favor of delivering cheap'n'easy for now. We first need to get traction. Once there is an ecosystem consuming RSS feeds it'll be much easier to define the most appropriate file format, necessary metadata etc etc. It's not like we were going with full vendor lock-in: RSS is standard, it would mostly be a matter of changing the base URL to switch to another server. On the other hand, going this way saves us the hassle of hosting a public server, generating RSS etc etc… and focus on value proposition definition 🙂

LucasVerneyDGE

comment created time in 7 days

GollumEvent
GollumEvent

Pull request review commentambanum/CGUs

Filter out country version blinking in Google Terms of Services (update)

 export function removeUTMfromUrls(document) {     link.href = link.href.replace(/utm_[^=]*=[^&]*[&]{0,1}/g, '');   }); }++export function removeCountryVersion(document) {+  const countryVersionLink = document.querySelector('a[data-name="country-version"]');+  const countryVersionTitle = document.querySelector('#footnote-country-version');+  const disclaimerTitle = document.querySelector('#footnote-disclaimer');++  if (countryVersionLink) {+    // Remove "country version" link at top of the page+    countryVersionLink.parentNode.remove();+  }++  if (countryVersionTitle && disclaimerTitle) {+    const countryVersionSectionContent = document.createRange();+    countryVersionSectionContent.setStartBefore(countryVersionTitle);+    countryVersionSectionContent.setEndBefore(disclaimerTitle);++    // Avoid removing too much content+    if (countryVersionSectionContent.cloneContents().querySelectorAll('h3').length > 1) {+      throw new Error('Country version selection seems to be too large.');

That's interesting. I'm not sure I fully understand the intention. Is it some sort of failsafe for unforeseen cases, or do we expect this condition to sometimes be matched?

Ndpnt

comment created time in 7 days

PullRequestReviewEvent

Pull request review commentambanum/CGUs

Filter out country version blinking in Google Terms of Services (update)

 export function removeUTMfromUrls(document) {     link.href = link.href.replace(/utm_[^=]*=[^&]*[&]{0,1}/g, '');   }); }++export function removeCountryVersion(document) {+  const countryVersionLink = document.querySelector('a[data-name="country-version"]');+  const countryVersionTitle = document.querySelector('#footnote-country-version');+  const disclaimerTitle = document.querySelector('#footnote-disclaimer');++  console.log('countryVersionLink', countryVersionLink);+  if (countryVersionLink) {+    // Remove "country version" link at top of the page+    countryVersionLink.parentNode.remove();+  }++  if (countryVersionTitle && disclaimerTitle) {+    // Remove all the "country version" section in the CGUs+    const countryVersionSectionContent = document.createRange();+    countryVersionSectionContent.setStartBefore(countryVersionTitle);

But we could just use assert instead of if (…) { throw } 🙂

Ndpnt

comment created time in 7 days

Pull request review commentambanum/CGUs

Filter out country version blinking in Google Terms of Services (update)

     "Terms of Service": {       "fetch": "https://policies.google.com/terms?hl=en-GB",       "filter": [-        "removeUTMfromUrls"+        "removeUTMfromUrls",+        "removeCountryVersion"
        "removeCountryVersionIfPresent"
Ndpnt

comment created time in 7 days

PullRequestReviewEvent

Pull request review commentambanum/CGUs

Fix services validation on CI

 let servicesToValidate = args.filter(arg => !arg.startsWith('--'));     run();   } catch (error) {     console.error(error);+    process.exit(1);

Why not just let the error be thrown, then? 🤔

Ndpnt

comment created time in 7 days

Pull request review commentambanum/CGUs

Fix services validation on CI

 jobs:      steps:     - uses: actions/checkout@v2+      with:+        fetch-depth: 0+        ref: ${{ github.event.pull_request.head.sha }}+    - run: git checkout HEAD^+    - run: git fetch origin master:master

Why do we have to always duplicate these instructions? Can't we just have both workflows chained?

Ndpnt

comment created time in 7 days

Pull request review commentambanum/CGUs

Fix services validation on CI

 jobs:      steps:     - uses: actions/checkout@v2+      with:+        fetch-depth: 0+        ref: ${{ github.event.pull_request.head.sha }}+    - run: git checkout HEAD^

Why do we go to the parent of HEAD instead of testing the last commit?

Ndpnt

comment created time in 7 days

PullRequestReviewEvent
PullRequestReviewEvent

issue openedambanum/CGUs

[Google] [ToS] Stop tracking images

Currently, images in base64 are stored while they are illustrative only. They could be removed to decrease the file size.

created time in 7 days

Pull request review commentambanum/CGUs

Limit attempts to restart to avoid looping on an unresolved error

  - name: Start CGUs   command:-    cmd: forever start -a --uid "{{ app }}" -c "npm run start:scheduler:prod" ./+    cmd: forever start -m 10 -a --uid "{{ app }}" -c "npm run start:scheduler:prod" ./

Maybe couple that with --spinSleepTime to decrease the probability of transient errors?

    cmd: forever start -m 10 --spinSleepTime 10000 -a --uid "{{ app }}" -c "npm run start:scheduler:prod" ./
Ndpnt

comment created time in 7 days

PullRequestReviewEvent
PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commentambanum/CGUs

Handle errors on services filters functions

 const consoleTransport = new winston.transports.Console(); const transports = [ consoleTransport ];  if (config.get('logger.sendMailOnError')) {-  transports.push(new winston.transports.Mail({-    level: 'error',+  const mailerOptions = {     to: config.get('logger.sendMailOnError.to'),     from: config.get('logger.sendMailOnError.from'),-    subject: '[CGUs] Error Report',     host: process.env.SMTP_HOST,     username: process.env.SMTP_USERNAME,     password: process.env.SMTP_PASSWORD,     ssl: true,     formatter: args => args[Object.getOwnPropertySymbols(args)[1]] // Returns the full error message, the same visible in the console. It is referenced in the argument object with a Symbol of which we do not have the reference but we know it is the second one.+  };++  transports.push(new winston.transports.Mail({+    ...mailerOptions,+    level: 'error',+    subject: '[CGUs] Error Report',+  }));++  transports.push(new winston.transports.Mail({+    ...mailerOptions,+    level: 'warn',+    subject: '[CGUs] Warning Report',

If a “warning” is always a “Content not accessible”, could we maybe:

    subject: '[CGUs] Inaccessible content',
Ndpnt

comment created time in 7 days

PullRequestReviewEvent
PullRequestReviewEvent

pull request commentambanum/CGUs

Drop Facebook redirections in Facebook.filters.js

Thanks @LucasVerneyDGE! The aim was to reduce noise by removing unique tracking IDs, but you're right with that extra removal!

In general, I think we should favor minimal interventions in filters in order to minimise the risk of not tracking the actual document content, but the reference of noise you gave combined with the sort of intervention justifies in my view this improvement 🙂

Do you agree with the general notion that we should minimise interventions? If so, I'll add it to the guidelines that will soon be consolidated from https://github.com/ambanum/CGUs/issues/106#issuecomment-681870747.

LucasVerneyDGE

comment created time in 13 days

push eventambanum/CGUs

Matti Schneider

commit sha a282cac5fb8ffe1885a7572bd01e7e3e1827480f

Remove unused variable in filter

view details

push time in 13 days

PR opened ambanum/CGUs

Update LibraryThing

Stop “Failed to execute 'setEndBefore' on 'Range': parameter 1 is not of type 'Node'.” errors.

+26 -33

0 comment

2 changed files

pr created time in 13 days

create barnchambanum/CGUs

branch : update-librarything

created branch time in 13 days

pull request commentambanum/TOSBack-CGUs-bridge

Continuation of https://github.com/ambanum/CGUs/pull/111

Yes, sorry! We were wondering how to make that more readable / detectable. Should we notify you when there are changes? Would SemVer be better? Shall we just have a chat when needed?

michielbdejong

comment created time in 18 days

pull request commentambanum/TOSBack-CGUs-bridge

Continuation of https://github.com/ambanum/CGUs/pull/111

Wow! Thanks @TosbackCgusBridge-Bot 😉

michielbdejong

comment created time in 18 days

pull request commentambanum/TOSBack-CGUs-bridge

Continuation of https://github.com/ambanum/CGUs/pull/111

beefy build server

🐄

Quite curious on the time it takes!

michielbdejong

comment created time in 18 days

push eventambanum/CGUs

Lucas Verney

commit sha 3bc17bdf60254be396d05fcbd69908515b4c3030

Improve documentation about configuration * Document missing / production configuration options. * Document environment variables. See https://github.com/ambanum/CGUs/pull/128#discussion_r486480869.

view details

Matti Schneider

commit sha 9502cfa0ab30ebd3c5fdc0f3992cf4023614a205

Add links

view details

Matti Schneider

commit sha f4bd324afb2424f56e43d11caf35d7658e5c4e82

Improve configuration documentation

view details

push time in 18 days

PR merged ambanum/CGUs

Improve documentation about configuration
  • Document missing / production configuration options.
  • Document environment variables.

See https://github.com/ambanum/CGUs/pull/128#discussion_r486480869.

+44 -0

0 comment

1 changed file

LucasVerneyDGE

pr closed time in 18 days

push eventLucasVerneyDGE/CGUs

Matti Schneider

commit sha 9502cfa0ab30ebd3c5fdc0f3992cf4023614a205

Add links

view details

push time in 18 days

Pull request review commentambanum/CGUs

Improve documentation about configuration

 The default configuration can be read and changed in `config/default.json`: } ``` +An example of a production configuration file can be found in `config/production.json`. It includes the extra configuration for:++* A logging mechanism, to be notified upon error (this requires a valid SMTP configuration, see environment variables below):++```json+{+  "logger": {+    "sendMailOnError": {+      "to": "recipient@example.com",+      "from": "cgu-bot@example.com"+    }+  }+}+```++* Public URLs to the snapshots and versions repositories, used to automate the initial database setup (`publicSnapshotsRepository` and `publicVersionsRepository` which are used by `npm setup`) and for the links in notifications (`snapshotsBaseUrl`):++```json+{+  "history": {+    "publicSnapshotsRepository": "https://github.com/ambanum/CGUs-snapshots.git",+    "publicVersionsRepository": "https://github.com/ambanum/CGUs-versions.git",+    "snapshotsBaseUrl": "https://github.com/ambanum/CGUs-snapshots/commit/"+  }+}+```++#### Environment variables++These environment variables can be provided in a `.env` file at the root of the repository. See `.env.example` for an example of such a file.++In order to be notified for errors, a valid SMTP configuration should be provided through the following environment variables:++* `SMTP_HOST` for the SMTP hostname+* `SMTP_USERNAME` / `SMTP_PASSWORD` for the credentials+++If your infrastructure requires using an outgoing HTTP/HTTPS proxy to access Internet, you can provide it through the `HTTP_PROXY` and `HTTPS_PROXY` environment variable.++In order to use the default SendInBlue-based notification mechanism, a valid API key should be provided through the `SENDINBLUE_API_KEY` environment variable.
In order to use the default [SendInBlue](https://www.sendinblue.com)-based notification mechanism, a valid API key should be provided through the `SENDINBLUE_API_KEY` environment variable.
LucasVerneyDGE

comment created time in 18 days

Pull request review commentambanum/CGUs

Improve documentation about configuration

 The default configuration can be read and changed in `config/default.json`: } ``` +An example of a production configuration file can be found in `config/production.json`. It includes the extra configuration for:++* A logging mechanism, to be notified upon error (this requires a valid SMTP configuration, see environment variables below):
* A logging mechanism, to be notified upon error (this requires a valid SMTP configuration, see [environment variables](#environment-variables) below):
LucasVerneyDGE

comment created time in 18 days

PullRequestReviewEvent
PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commentambanum/CGUs

Change the recipient of error emails

   },   "logger": {     "sendMailOnError": {-      "to": "nicolas.dupont@beta.gouv.fr",+      "to": "matti.schneider@beta.gouv.fr",

Lovely, thanks for offering! Any help on improving the documentation would be welcome 😃

We should have updated the “Configuring” section of the README in #112 to reflect this new option 😞 This is where this documentation should go.

If you could document the environment variables there as well, perhaps splitting both parts in subtitles (#### Config file / #### Environment variables) and documenting the HTTP*_PROXY environment variables you added in #32, it would be awesome!

Ndpnt

comment created time in 19 days

issue commentambanum/CGUs

Only notify for a subset of services

requiring a SendInBlue to be able to deploy CGUs with notifications could be a quite strong requirement

Totally, which is why we've opted for an events-based architecture and a Notifier class which is fully independent from the rest of the code and could easily be extracted in the future 🙂

LucasVerneyDGE

comment created time in 19 days

delete branch ambanum/CGUs

delete branch : ops

delete time in 19 days

PR merged ambanum/CGUs

Improve ops documentation
+11 -3

0 comment

1 changed file

MattiSG

pr closed time in 19 days

push eventambanum/CGUs

Matti Schneider

commit sha f901805b3efe382c59c5108a2e7ecfafa8ef07c3

Use long options in ops documentation

view details

Matti Schneider

commit sha 3f4cf17e0ad2936f5bc3b705682d7e2e99ab71bb

Document how to obtain logs

view details

Matti Schneider

commit sha e9570d213aed9b40e45ade3cc81081862daeb0a6

Improve ops documentation

view details

push time in 19 days

push eventambanum/CGUs

Nicolas Dupont

commit sha d6dad97dfb276e74c64a3f3e9907a08013c5a2ea

Ignore unknown services

view details

Matti Schneider

commit sha bce968c45ecd349268f1b24d2d417fc1029800b0

Remove unnecessary check

view details

Nicolas Dupont

commit sha 076589eb25469564cd2120afd82f9a75fcd85176

Ignore unknown services Ignore unknown services

view details

Nicolas Dupont

commit sha 1f64122fbb6b8dbc6ead76dfc2b52ea6f26618a8

Change the recipient of error emails

view details

Matti Schneider

commit sha 5ff8b126038812798c95f4296bffee3ec4410382

Change the recipient of error emails

view details

Nicolas Dupont

commit sha 2e09ed6cb70e1dce00de3d51df0e86a3bc9d22f7

Get file path and commit id in one git command Improve perf x3

view details

Nicolas Dupont

commit sha 572f3d99a1f7e6abc0c1088ebdad9abbd8cedfd1

Remove dead code

view details

Nicolas Dupont

commit sha 1a6242502a7d8ea082b333d74e6c7e78f85e7d57

Improve function name

view details

Matti Schneider

commit sha 29834cc83c332569ff4ade9b05bb19546940e243

Fix typo

view details

Nicolas Dupont

commit sha 689799d7c386e33314e79a30da92effe6ddc788e

Improve refiltering performance Improve refiltering performance

view details

Matti Schneider

commit sha f901805b3efe382c59c5108a2e7ecfafa8ef07c3

Use long options in ops documentation

view details

Matti Schneider

commit sha 3f4cf17e0ad2936f5bc3b705682d7e2e99ab71bb

Document how to obtain logs

view details

push time in 19 days

issue commentambanum/CGUs

Only notify for a subset of services

RSS is an excellent idea, thanks a lot for suggesting this! 😃

I believe this could be a standard way to expose such functionality with limited cost on our side, by allowing the entire RSS ecosystem to plug into CGUs.

Would SendInBlue offer a similar tool?

Yes! Which demonstrates even more the relevance of this implementation 😃

We'll still need to create a public domain for this feature to be fully usable, but we can definitely start work on the file format, for example.

Would you feel like suggesting an XML schema for the output?

LucasVerneyDGE

comment created time in 19 days

push eventambanum/CGUs

Nicolas Dupont

commit sha 1f64122fbb6b8dbc6ead76dfc2b52ea6f26618a8

Change the recipient of error emails

view details

Matti Schneider

commit sha 5ff8b126038812798c95f4296bffee3ec4410382

Change the recipient of error emails

view details

Nicolas Dupont

commit sha 2e09ed6cb70e1dce00de3d51df0e86a3bc9d22f7

Get file path and commit id in one git command Improve perf x3

view details

Nicolas Dupont

commit sha 572f3d99a1f7e6abc0c1088ebdad9abbd8cedfd1

Remove dead code

view details

Nicolas Dupont

commit sha 1a6242502a7d8ea082b333d74e6c7e78f85e7d57

Improve function name

view details

Matti Schneider

commit sha 29834cc83c332569ff4ade9b05bb19546940e243

Fix typo

view details

push time in 19 days

push eventambanum/CGUs

Nicolas Dupont

commit sha 1f64122fbb6b8dbc6ead76dfc2b52ea6f26618a8

Change the recipient of error emails

view details

Matti Schneider

commit sha 5ff8b126038812798c95f4296bffee3ec4410382

Change the recipient of error emails

view details

push time in 19 days

delete branch ambanum/CGUs

delete branch : change-the-recipient-of-error-emails

delete time in 19 days

PR opened ambanum/CGUs

Reviewers
Improve ops documentation
+11 -3

0 comment

1 changed file

pr created time in 19 days

PullRequestReviewEvent

create barnchambanum/CGUs

branch : ops

created branch time in 19 days

PR closed ambanum/CGUs

Add stats npm command
+14 -1

1 comment

2 changed files

Ndpnt

pr closed time in 19 days

pull request commentambanum/CGUs

Add stats npm command

Just saw this is already provided since #112 as the first line when running npm start:

2020-09-10 17:23:46 info                                                          Refiltering 136 documents from 51 services…

I'm not sure this is worth adding dedicated code with no other use case than human extraction 🙂 Please re-open if I missed a use case!

Ndpnt

comment created time in 19 days

PullRequestReviewEvent

Pull request review commentambanum/CGUs

Ignore unknown services

 let serviceIds = args.filter(arg => !arg.startsWith('--'));   app.attach(logger);   await app.init(); +  if (serviceIds.length) {

Not needed.

Ndpnt

comment created time in 19 days

more