profile
viewpoint
Matias Insaurralde matiasinsaurralde @TykTechnologies Paraguay https://matias.insaurral.de/ software developer (most of the time), infosec enthusiast, human being, self-taught. 24 years.

apla/atom-jscad 28

Previewing (J)SCAD 3D models inside Atom.

cmelgarejo/osk_login 3

Automate login and parsing of homebanking accounts, get the needed information like balance, debts, etc and logout.

asoorm/serverless 2

Drivers for serverless providers.

carloscarvallo/React-Gateway 2

HTTP request logger in React

matiasinsaurralde/apbp 2

Another PHP Blog's Planetarium.

matiasinsaurralde/atom-scad-preview 2

Previewing (J)SCAD 3D models inside Atom.

carloscarvallo/mini-go-wit 1

Example of Wit.ai API integrated with Messenger API

matiasinsaurralde/ach 1

ACH(Automated Clearing House) GoLang library implementing NACHA file creation and validation via reader and writer

push eventmatiasinsaurralde/tyk

Furkan Senharputlu

commit sha 49161a34914ccf1cfdd4408de1cf1b3ac6cc6591

Fix trace request builder func (#3262) When `tr.Path` was empty, it was giving error. So I change `httptest.NewRequest()` to `http.NewRequest()`. Fixes https://github.com/TykTechnologies/tyk-analytics/issues/2031

view details

Furkan Senharputlu

commit sha c4bb51405b362a1abfc968b750c3e3fc7b6380f3

Add test for loading blob and file templates (#3267) Related to https://github.com/TykTechnologies/tyk/issues/2518

view details

Sredny M

commit sha 273ba6eb66336a0517d2d4d4eedb1fb34c74dcdf

when rpc in emergency mode then dont check org session in rpc (#3163)

view details

Alok G Singh

commit sha 6b46b60f654a114ceaf67a05bcc6ed6211d5984a

Syncing integration workflow from tyk-ci (#3266) Now using the dedicated subaccount for devenvs

view details

push time in 2 days

startedTykTechnologies/gromit

started time in 2 days

push eventTykTechnologies/tyk

Sredny M

commit sha 273ba6eb66336a0517d2d4d4eedb1fb34c74dcdf

when rpc in emergency mode then dont check org session in rpc (#3163)

view details

push time in 2 days

delete branch TykTechnologies/tyk

delete branch : fix-3162

delete time in 2 days

PR merged TykTechnologies/tyk

Reviewers
when rpc in emergency mode then dont check org session in rpc

<!-- Provide a general summary of your changes in the Title above -->

Description

When a slave gw has enabled analytics, and sink is down, then dont try to get org session from rpc

Related Issue

https://github.com/TykTechnologies/tyk/issues/3162

Motivation and Context

Give solution to https://github.com/TykTechnologies/tyk/issues/3162 also, fix how gw behaves when sink is down and do not affect the response time for the apis.

How This Has Been Tested

  • Setup MDCB environment and ensure that slaved gw have enable_analytics set to true
  • Consume api in master and slaved gw
  • Turn off sink
  • Consume API in slaved gw, the time of response should be similar as step 2 (however, consider that it can take some seconds to change the state to emergency mode)
  • Consume api in master (to ensure that nothing is broken now)
  • Turn on again sink
  • Consume api in slave

Some load test were performed using apache benchmark sending 10 request in parallel and 100 request in total (executing ab -n 100 -c 10 http://tyk-gateway:8182/1/ ), next are the results after the change:

1- With Sink up&running:

Requests per second:    13.91 [#/sec] (mean)
Time per request:       718.859 [ms] (mean) (this is for a batch of 10 requests)
Time per request:       71.886 [ms] (mean, across all concurrent requests) (per single request)

2- With Sink down:

Requests per second:    13.79 [#/sec] (mean)
Time per request:       725.245 [ms] (mean)
Time per request:       72.524 [ms] (mean, across all concurrent requests)

As we see, the number are almost the same. And just as a reference these are the numbers with sink down before make this change:

Requests per second:    0.97 [#/sec] (mean)
Time per request:       10302.129 [ms] (mean)
Time per request:       1030.213 [ms] (mean, across all concurrent requests)

Screenshots (if appropriate)

Types of changes

<!-- What types of changes does your code introduce? Put an x in all the boxes that apply: -->

  • [x] Bug fix (non-breaking change which fixes an issue)
  • [ ] New feature (non-breaking change which adds functionality)
  • [ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist

<!-- Go over all the following points, and put an x in all the boxes that apply --> <!-- If you're unsure about any of these, don't hesitate to ask; we're here to help! -->

  • [x] Make sure you are requesting to pull a topic/feature/bugfix branch (right side). If pulling from your own fork, don't request your master!
  • [x] Make sure you are making a pull request against the master branch (left side). Also, you should start your branch off our latest master.
  • [ ] My change requires a change to the documentation.
    • [ ] If you've changed APIs, describe what needs to be updated in the documentation.
    • [ ] If new config option added, ensure that it can be set via ENV variable
  • [ ] I have updated the documentation accordingly.
  • [ ] Modules and vendor dependencies have been updated; run go mod tidy && go mod vendor
  • [ ] When updating library version must provide reason/explanation for this update.
  • [ ] I have added tests to cover my changes.
  • [x] All new and existing tests passed.
  • [x] Check your code additions will not fail linting checks:
    • [x] go fmt -s
    • [x] go vet
+43 -1

3 comments

3 changed files

sredxny

pr closed time in 2 days

create barnchmatiasinsaurralde/tyk-identity-broker

branch : release-1.0

created branch time in 2 days

push eventmatiasinsaurralde/tyk-identity-broker

sredny buitrago

commit sha d7f4afc999b3f81e2dc27a099f83653fed480c0f

on http client, set skip verify as well

view details

sredny buitrago

commit sha 7afdf866899ab7cc30fb7f7654339b47c6fe9d6b

clean code

view details

sredny buitrago

commit sha 9ffce992806a9a8c2bb7948ea632ad03db9a70b5

added field to save profle name

view details

sredny buitrago

commit sha 11fb8b785d28729dac94dbe37c88eb0bd8a08831

TT-218 gofmt files, go import files

view details

Matias Insaurralde

commit sha e694e89dbfd013f24dd5e3bf604cbd00c1a8eb79

Merge pull request #127 from TykTechnologies/TT-218-add-name-field-for-profile added field to save profle name

view details

Matias Insaurralde

commit sha be8774ee2b50449b0fba3a1e602916c5b2cea30d

Merge pull request #126 from TykTechnologies/allow-skiptlsverify-consuming-dash-api skipTLSVerify while consuming dashboard api {do not merge}

view details

push time in 2 days

push eventTykTechnologies/tyk-identity-broker

sredny buitrago

commit sha d7f4afc999b3f81e2dc27a099f83653fed480c0f

on http client, set skip verify as well

view details

sredny buitrago

commit sha 7afdf866899ab7cc30fb7f7654339b47c6fe9d6b

clean code

view details

Matias Insaurralde

commit sha be8774ee2b50449b0fba3a1e602916c5b2cea30d

Merge pull request #126 from TykTechnologies/allow-skiptlsverify-consuming-dash-api skipTLSVerify while consuming dashboard api {do not merge}

view details

push time in 2 days

PR merged TykTechnologies/tyk-identity-broker

Reviewers
skipTLSVerify while consuming dashboard api {do not merge}

<!-- Provide a general summary of your changes in the Title above -->

Description

for POCs is need sometimes to set skipTLSVerify:true and therefore have the ability to use self signed certificates.

Related Issue

https://github.com/TykTechnologies/tyk-identity-broker/issues/102

Motivation and Context

Give solution to https://github.com/TykTechnologies/tyk-identity-broker/issues/102

How This Has Been Tested

  • Ran dashboard and TIB in HTTPS
  • in TIB config file set config.HttpServerOptions.SSLInsecureSkipVerify:true
  • Call TIB with `curl --location -vk --request POST 'https://localhost:3010/auth/ldap-for-sso-dashboard/ADProvider?username=read-only-admin&password=password'
  • Nonce is created and a URL to login to dev portal comes in the response

Screenshots (if appropriate)

Response:

*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 3010 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: C=co; ST=bgta; L=bgta; OU=tyk-dashboard; CN=tyk-dashboard; emailAddress=sredny@tyk.io
*  start date: Aug 10 20:10:19 2020 GMT
*  expire date: Aug  8 20:10:19 2030 GMT
*  issuer: C=co; ST=bgta; L=bgta; OU=tyk-dashboard; CN=tyk-dashboard; emailAddress=sredny@tyk.io
*  SSL certificate verify result: self signed certificate (18), continuing anyway.
> POST /auth/ldap-for-sso-dashboard/ADProvider?username=read-only-admin&password=password HTTP/1.1
> Host: localhost:3010
> User-Agent: curl/7.64.1
> Accept: */*
> 
< HTTP/1.1 301 Moved Permanently
< Location: http://tyk-portal:3000/sso/?nonce=MTAyZGI1YTUtNmQwZC00ZWJiLTdlODMtODQ5YThlYWY1NDkz
< Date: Mon, 10 Aug 2020 21:46:34 GMT
< Content-Length: 0

Types of changes

<!-- What types of changes does your code introduce? Put an x in all the boxes that apply: -->

  • [x] Bug fix (non-breaking change which fixes an issue)
  • [ ] New feature (non-breaking change which adds functionality)
  • [ ] Breaking change (fix or feature that would cause existing functionality to change)
  • [ ] Refactoring or add test (improvements in base code or adds test coverage to functionality)

Checklist

<!-- Go over all the following points, and put an x in all the boxes that apply --> <!-- If you're unsure about any of these, don't hesitate to ask; we're here to help! -->

  • [x] Make sure you are requesting to pull a topic/feature/bugfix branch (right side). If pulling from your own fork, don't request your master!
  • [x] Make sure you are making a pull request against the master branch (left side). Also, you should start your branch off our latest master.
  • [ ] My change requires a change to the documentation.
    • [ ] If you've changed APIs, describe what needs to be updated in the documentation.
    • [ ] If new config option added, ensure that it can be set via ENV variable
  • [ ] I have updated the documentation accordingly.
  • [ ] Modules and vendor dependencies have been updated; run go mod tidy && go mod vendor
  • [ ] When updating library version must provide reason/explanation for this update.
  • [ ] I have added tests to cover my changes.
  • [x] All new and existing tests passed.
  • [x] Check your code additions will not fail linting checks:
    • [x] go fmt -s
    • [x] go vet
+13 -7

0 comment

2 changed files

sredxny

pr closed time in 2 days

push eventTykTechnologies/tyk-identity-broker

sredny buitrago

commit sha 9ffce992806a9a8c2bb7948ea632ad03db9a70b5

added field to save profle name

view details

sredny buitrago

commit sha 11fb8b785d28729dac94dbe37c88eb0bd8a08831

TT-218 gofmt files, go import files

view details

Matias Insaurralde

commit sha e694e89dbfd013f24dd5e3bf604cbd00c1a8eb79

Merge pull request #127 from TykTechnologies/TT-218-add-name-field-for-profile added field to save profle name

view details

push time in 2 days

PR merged TykTechnologies/tyk-identity-broker

Reviewers
added field to save profle name

<!-- Provide a general summary of your changes in the Title above -->

Description

Added field for Names to each profile. String and optional

Related Issue

https://tyktech.atlassian.net/browse/TT-218

Motivation and Context

Add a Name to profiles so FE can display them in a pretty way

How This Has Been Tested

CRUD profile

Screenshots (if appropriate)

Types of changes

<!-- What types of changes does your code introduce? Put an x in all the boxes that apply: -->

  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [x] New feature (non-breaking change which adds functionality)
  • [ ] Breaking change (fix or feature that would cause existing functionality to change)
  • [ ] Refactoring or add test (improvements in base code or adds test coverage to functionality)

Checklist

<!-- Go over all the following points, and put an x in all the boxes that apply --> <!-- If you're unsure about any of these, don't hesitate to ask; we're here to help! -->

  • [x] Make sure you are requesting to pull a topic/feature/bugfix branch (right side). If pulling from your own fork, don't request your master!
  • [x] Make sure you are making a pull request against the master branch (left side). Also, you should start your branch off our latest master.
  • [ ] My change requires a change to the documentation.
    • [ ] If you've changed APIs, describe what needs to be updated in the documentation.
    • [ ] If new config option added, ensure that it can be set via ENV variable
  • [ ] I have updated the documentation accordingly.
  • [ ] Modules and vendor dependencies have been updated; run go mod tidy && go mod vendor
  • [ ] When updating library version must provide reason/explanation for this update.
  • [ ] I have added tests to cover my changes.
  • [x] All new and existing tests passed.
  • [x] Check your code additions will not fail linting checks:
    • [x] go fmt -s
    • [x] go vet
+2 -1

0 comment

1 changed file

sredxny

pr closed time in 2 days

fork matiasinsaurralde/go-memdev

Golang memory devices information :package:

fork in 4 days

startedbitcav/go-memdev

started time in 4 days

push eventmatiasinsaurralde/tyk-identity-broker

sredny buitrago

commit sha ce62d6bfb4a0f6ba971cab56b9d9e76f13b705e3

fix typo in import

view details

sredny buitrago

commit sha 192d0538c13663d69849b72c859493d2ac67dbb9

Merge branch 'master' of https://github.com/TykTechnologies/tyk-identity-broker

view details

Matias Insaurralde

commit sha 1f5de71addee218f20d53fc4b61e5a40d43d4f1a

Merge pull request #124 from TykTechnologies/fix-typo-importing-Initializer Fix typo importing initializer

view details

Alok G Singh

commit sha cd2b422611c5b57ca1e96d9a8ae13bfc7c8aa407

Use new signing key

view details

Leonid Bugaev

commit sha 3e1aff41466d4981b984f9b99e6106eae18c69dc

Merge pull request #125 from TykTechnologies/as/new-key #Description Also use the gpg infra that the other pipelines use. ## Motivation and Context Problem uncovered when building release-0.7. ## How This Has Been Tested Full [run](https://app.buddy.works/tyk-projects/tyk-identity-broker/pipelines/pipeline/171074/execution/5f318f89da64d3331fc487de) of unstable pipeline. ## Types of changes <!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: --> - [x] Bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Refactoring or add test (improvements in base code or adds test coverage to functionality) ## Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply --> <!-- If you're unsure about any of these, don't hesitate to ask; we're here to help! --> - [x] Make sure you are requesting to **pull a topic/feature/bugfix branch** (right side). If pulling from your own fork, don't request your `master`! - [x] Make sure you are making a pull request against the **`master` branch** (left side). Also, you should start *your branch* off *our latest `master`*. - [ ] My change requires a change to the documentation. - [ ] If you've changed APIs, describe what needs to be updated in the documentation. - [ ] If new config option added, ensure that it can be set via ENV variable - [ ] I have updated the documentation accordingly. - [ ] Modules and vendor dependencies have been updated; run `go mod tidy && go mod vendor` - [ ] When updating library version must provide reason/explanation for this update. - [ ] I have added tests to cover my changes. - [x] All new and existing tests passed. - [ ] Check your code additions will not fail linting checks: - [ ] `go fmt -s` - [ ] `go vet`

view details

push time in 5 days

created tagTykTechnologies/tyk-pump

tagv1.0.1

Tyk Analytics Pump to move analytics data from Redis to any supported back end (multiple back ends can be written to at once).

created time in 5 days

created tagmatiasinsaurralde/tyk-identity-broker

tagv0.7.2

Tyk Authentication Proxy for third-party login

created time in 5 days

created tagmatiasinsaurralde/tyk-pump

tagv1.0.1

Tyk Analytics Pump to move analytics data from Redis to any supported back end (multiple back ends can be written to at once).

created time in 5 days

startedsecuresystemslab/zippy

started time in 5 days

push eventTykTechnologies/tyk-identity-broker

sredny buitrago

commit sha ce62d6bfb4a0f6ba971cab56b9d9e76f13b705e3

fix typo in import

view details

sredny buitrago

commit sha 192d0538c13663d69849b72c859493d2ac67dbb9

Merge branch 'master' of https://github.com/TykTechnologies/tyk-identity-broker

view details

Matias Insaurralde

commit sha 1f5de71addee218f20d53fc4b61e5a40d43d4f1a

Merge pull request #124 from TykTechnologies/fix-typo-importing-Initializer Fix typo importing initializer

view details

push time in 8 days

PR merged TykTechnologies/tyk-identity-broker

Fix typo importing initializer

<!-- Provide a general summary of your changes in the Title above -->

Description

In main.go we import the Initializer, this package was renamed but the import didn't, so, in this PR that is fixed

Related Issue

None. Build issue in Buddy

Motivation and Context

Build TIB in Buddy

How This Has Been Tested

<!-- Please describe in detail how you tested your changes --> <!-- Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. -->

Screenshots (if appropriate)

Types of changes

<!-- What types of changes does your code introduce? Put an x in all the boxes that apply: -->

  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [ ] New feature (non-breaking change which adds functionality)
  • [ ] Breaking change (fix or feature that would cause existing functionality to change)
  • [ ] Refactoring or add test (improvements in base code or adds test coverage to functionality)

Checklist

<!-- Go over all the following points, and put an x in all the boxes that apply --> <!-- If you're unsure about any of these, don't hesitate to ask; we're here to help! -->

  • [ ] Make sure you are requesting to pull a topic/feature/bugfix branch (right side). If pulling from your own fork, don't request your master!
  • [ ] Make sure you are making a pull request against the master branch (left side). Also, you should start your branch off our latest master.
  • [ ] My change requires a change to the documentation.
    • [ ] If you've changed APIs, describe what needs to be updated in the documentation.
    • [ ] If new config option added, ensure that it can be set via ENV variable
  • [ ] I have updated the documentation accordingly.
  • [ ] Modules and vendor dependencies have been updated; run go mod tidy && go mod vendor
  • [ ] When updating library version must provide reason/explanation for this update.
  • [ ] I have added tests to cover my changes.
  • [ ] All new and existing tests passed.
  • [ ] Check your code additions will not fail linting checks:
    • [ ] go fmt -s
    • [ ] go vet
+1 -2

0 comment

1 changed file

sredxny

pr closed time in 8 days

push eventmatiasinsaurralde/tyk-identity-broker

vesko-tyk

commit sha 830fc060e49ef69a784545fe146a3f55d0ed2fb2

Update docker image (#72)

view details

Leonid Bugaev

commit sha 19732df4e6ddc5a113c78fa2a86aaccb5aee2b5e

Allow configure OpenID scopes via `Scopes` string array (#74) Fix https://github.com/TykTechnologies/tyk-identity-broker/issues/75

view details

sredny

commit sha 8831f2be2122859149b1142ed133b45a40af5aa4

created package for data loader,implemented mongo loader

view details

sredny

commit sha d9c889eef775991388aeee8e507f08d43243d5a9

implemented support for mongo as dataLoader

view details

sredny

commit sha 47c70fc4c10c387ba822206270a0adba934e9fdd

Merge branch 'master' into support-for-mongo-db

view details

sredny

commit sha e3fe699c2be75aa567b192c2724e34d170b67f56

removed endpoint to flush, for mongo backups we only use one collection. Wrote test for Create Mongo Loader

view details

Matias Insaurralde

commit sha e7200df98c7bd51fd1d45cd2ff7f8ec67b688552

Fix Redis SSL fields (#78) Redis SSL fields aren't populated when calling `newRedisClusterPool`, this fixes the issue, the following configuration now works: ```json { "Secret": "test-secret", "HttpServerOptions": { "UseSSL": false, "CertFile": "./certs/server.pem", "KeyFile": "./certs/server.key" }, "BackEnd": { "Name": "redis", "ProfileBackendSettings": {}, "IdentityBackendSettings": { "Hosts" : { "localhost": "6379" }, "UseSSL": true, "SSLInsecureSkipVerify": true, "Password": "testpassword", "Database": 0, "EnableCluster": false, "MaxIdle": 1000, "MaxActive": 2000 } }, "TykAPISettings": { "GatewayConfig": { "Endpoint": "http://localhost", "Port": "80", "AdminSecret": "54321" }, "DashboardConfig": { "Endpoint": "http://localhost", "Port": "3000", "AdminSecret": "12345" } } } ```

view details

Leonid Bugaev

commit sha 0d14431c09849742ee91bc9171172f32ed34eb26

Add support for specifying user group id (#76) You can set static value via `DefaultGroupID` or dynamic value based on field of oAuth/OpenID scope using `CustomUserGroupField` and `UserGroupMapping` fields. Example: ``` { "DefaultUserGroupID": "default-user-group", "CustomUserGroupField": "scope", "UserGroupMapping": { "admin": "<admin-group-id>", "analytics": "<analytics-group-id>" } } ```

view details

Mark Southee

commit sha 7d80e7a8d5cff495d4f829a15ad6bab6e5ff1b64

Change LDAPUseSSL to true (#69) * Change LDAPUseSSL to true * Further updates

view details

sredny

commit sha 1f0bf9fc9b6c8baac61154a3abc8614af4be8473

refactored basic actions for profiles in order that can be consumed as methods from external apps

view details

Lanre Adelowo

commit sha b158e95c3558cf3e63a8cc79574501f3c66b67d1

add prevention against race conditions (#79)

view details

sredny

commit sha db969cbe2d87d6e7396d6a463f0d8a9e90f788cd

started to separated logic of oauth flow endpoints

view details

sredny

commit sha 1a1a80873c74fea2d565aac4b0fe4f5218e340ab

separated base logic for oauth flow in order that can be used by dashboard

view details

sredny

commit sha e7724360d77ce96c1f72bfa85a8b8b3473b7e528

reestructured code to work as library

view details

sredny

commit sha 8281fc621363db8b063f741a048b9566f41d19eb

reestructured and continue with separation of logic in order that it can be used as a library

view details

Leonid Bugaev

commit sha 665e6af175655215b369ccfba4c4f4fdbbf4dbdd

Fill rawData attributes for LDAP

view details

Leonid Bugaev

commit sha 4c267aa0223556077a0a7349a8aa03e847e137f8

Fill rawData attributes for LDAP (#81) Part of https://github.com/TykTechnologies/tyk-identity-broker/issues/32

view details

joshblakeley

commit sha 9d6b354089d4c203420720243911dd890a42f699

initialise user rawdata and a few typos fixed

view details

Josh Blakeley

commit sha eafbfae44b8736b1bc4138c6b54041f6f04cbbd8

Merge branch 'master' into initialise-usermap

view details

Leonid Bugaev

commit sha c109c03ed833eff8a4f2fa8370e92ac4ca7925f2

Merge pull request #82 from TykTechnologies/initialise-usermap initialise user rawdata and a few typos fixed

view details

push time in 8 days

push eventTykTechnologies/tyk-identity-broker

sredny buitrago

commit sha 0a4e7d4f61551ce77dd8d9ecd63c0cdd223192c3

update vendor, updated version from 1.0 to 1.0.0

view details

sredny buitrago

commit sha 1ee4ee5c0516c62e6f7626bbb6e63c81b01ed08a

added gw certs to vendor

view details

Matias Insaurralde

commit sha ff4908ff4eca323a0db02ce8f2f98b7aac8c8ad4

Merge pull request #123 from TykTechnologies/update-vendor update vendor, updated version from 1.0 to 1.0.0

view details

push time in 8 days

PR merged TykTechnologies/tyk-identity-broker

update vendor, updated version from 1.0 to 1.0.0

<!-- Provide a general summary of your changes in the Title above -->

Description

Update vendor and version from 1.0 to 1.0.0

Related Issue

<!-- This project only accepts pull requests related to open issues --> <!-- If suggesting a new feature or change, please discuss it in an issue first --> <!-- If fixing a bug, there should be an issue describing it with steps to reproduce --> <!-- Please link to the issue here -->

Motivation and Context

<!-- Why is this change required? What problem does it solve? -->

How This Has Been Tested

<!-- Please describe in detail how you tested your changes --> <!-- Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. -->

Screenshots (if appropriate)

Types of changes

<!-- What types of changes does your code introduce? Put an x in all the boxes that apply: -->

  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [ ] New feature (non-breaking change which adds functionality)
  • [ ] Breaking change (fix or feature that would cause existing functionality to change)
  • [ ] Refactoring or add test (improvements in base code or adds test coverage to functionality)

Checklist

<!-- Go over all the following points, and put an x in all the boxes that apply --> <!-- If you're unsure about any of these, don't hesitate to ask; we're here to help! -->

  • [ ] Make sure you are requesting to pull a topic/feature/bugfix branch (right side). If pulling from your own fork, don't request your master!
  • [ ] Make sure you are making a pull request against the master branch (left side). Also, you should start your branch off our latest master.
  • [ ] My change requires a change to the documentation.
    • [ ] If you've changed APIs, describe what needs to be updated in the documentation.
    • [ ] If new config option added, ensure that it can be set via ENV variable
  • [ ] I have updated the documentation accordingly.
  • [ ] Modules and vendor dependencies have been updated; run go mod tidy && go mod vendor
  • [ ] When updating library version must provide reason/explanation for this update.
  • [ ] I have added tests to cover my changes.
  • [ ] All new and existing tests passed.
  • [ ] Check your code additions will not fail linting checks:
    • [ ] go fmt -s
    • [ ] go vet
+15705 -512079

0 comment

1259 changed files

sredxny

pr closed time in 8 days

push eventTykTechnologies/tyk-identity-broker

sredny buitrago

commit sha 59d0837ce11b6b1a6dd66d7119f0bd1b0f3793ce

allow dashboard to set custom super disptacher

view details

sredny buitrago

commit sha 4e4fb743c5b52e937f436a824a7c53d7388eae53

on error creating request to dashboard then dont panic and handle error

view details

Matias Insaurralde

commit sha 3d24df1de5eea1470c9a014d709d0f81ab9be1bc

Merge pull request #120 from TykTechnologies/ability-t-receive-custom-super-dashboard-dispatch internal tib - Ability to receive custom super-dashboard-dispatcher

view details

push time in 8 days

PR merged TykTechnologies/tyk-identity-broker

Reviewers
internal tib - Ability to receive custom super-dashboard-dispatcher

<!-- Provide a general summary of your changes in the Title above -->

Description

For embeded TIB is required to pass as well a request dispatcher for admin dashboard api, this api is consumed when the request fail to the standard api. Also handled the error throwed when is attempted to build a request without all the required parameters, in this way it will not panic.

Related Issue

  • https://github.com/TykTechnologies/tyk-identity-broker/issues/119
  • https://github.com/TykTechnologies/tyk-analytics/issues/2051
  • https://github.com/TykTechnologies/tyk-analytics/issues/2028

Motivation and Context

Give solution to:

  • https://github.com/TykTechnologies/tyk-identity-broker/issues/119
  • https://github.com/TykTechnologies/tyk-analytics/issues/2051
  • https://github.com/TykTechnologies/tyk-analytics/issues/2028

How This Has Been Tested

  • In testing

Screenshots (if appropriate)

Types of changes

<!-- What types of changes does your code introduce? Put an x in all the boxes that apply: -->

  • [x] Bug fix (non-breaking change which fixes an issue)
  • [ ] New feature (non-breaking change which adds functionality)
  • [ ] Breaking change (fix or feature that would cause existing functionality to change)
  • [ ] Refactoring or add test (improvements in base code or adds test coverage to functionality)

Checklist

<!-- Go over all the following points, and put an x in all the boxes that apply --> <!-- If you're unsure about any of these, don't hesitate to ask; we're here to help! -->

  • [x] Make sure you are requesting to pull a topic/feature/bugfix branch (right side). If pulling from your own fork, don't request your master!
  • [x] Make sure you are making a pull request against the master branch (left side). Also, you should start your branch off our latest master.
  • [ ] My change requires a change to the documentation.
    • [ ] If you've changed APIs, describe what needs to be updated in the documentation.
    • [ ] If new config option added, ensure that it can be set via ENV variable
  • [ ] I have updated the documentation accordingly.
  • [ ] Modules and vendor dependencies have been updated; run go mod tidy && go mod vendor
  • [ ] When updating library version must provide reason/explanation for this update.
  • [ ] I have added tests to cover my changes.
  • [x] All new and existing tests passed.
  • [x] Check your code additions will not fail linting checks:
    • [x] go fmt -s
    • [x] go vet
+9 -3

0 comment

1 changed file

sredxny

pr closed time in 8 days

push eventTykTechnologies/tyk-identity-broker

sredny buitrago

commit sha 90faeca0a4abcd0d33f58cfea8c571f17f1d6b0d

implement cert getter for certs

view details

sredny buitrago

commit sha 2c5a3adbffbd3635488338727e07391cd45b26bc

initialize cert default loader before its set

view details

sredny buitrago

commit sha f45b4e5acb1cfa1402fd3083404ad9023f4eb4fb

fmt files

view details

sredny buitrago

commit sha 4a14978d98307b030f6217db3f29273e50b2c441

for SAML, read cert location instead different paths for cert and key

view details

Matias Insaurralde

commit sha 8d3cf1dd8c39532493b64e72c9789abe791f59e3

Merge pull request #118 from TykTechnologies/connect-to-cert-manager implement cert getter for SAML certs

view details

push time in 8 days

PR merged TykTechnologies/tyk-identity-broker

Reviewers
implement cert getter for SAML certs

<!-- Provide a general summary of your changes in the Title above -->

Description

change the way how the certs are loaded, these are the certs used in SAML. Basically, not depend only in the path where the cert live, but also provide a different way (redis) that doesn't depends in the cert path, as in some environments the users doens't have access to the server to put their files

Related Issue

https://github.com/TykTechnologies/tyk-analytics/issues/2044

Motivation and Context

give solution to https://github.com/TykTechnologies/tyk-analytics/issues/2044

How This Has Been Tested

  • Ran TIB as an external service
  • Created profile for SAML but setting the field ProviderConfig.CertLocation to point to a path in the OS file system
  • Consume profile. Everything is ok
  • Ran TIB as internal tib
  • From dashboard upload a cert that contains the private key in the same file
  • Go to Identity manager and create a profile for SAML
  • Set the value for field ProviderConfig.CertLocation the ID of the cert
  • Execute flow with the profile, and everything went well

Screenshots (if appropriate)

Types of changes

<!-- What types of changes does your code introduce? Put an x in all the boxes that apply: -->

  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [x] New feature (non-breaking change which adds functionality)
  • [ ] Breaking change (fix or feature that would cause existing functionality to change)
  • [ ] Refactoring or add test (improvements in base code or adds test coverage to functionality)

Checklist

<!-- Go over all the following points, and put an x in all the boxes that apply --> <!-- If you're unsure about any of these, don't hesitate to ask; we're here to help! -->

  • [x] Make sure you are requesting to pull a topic/feature/bugfix branch (right side). If pulling from your own fork, don't request your master!
  • [x] Make sure you are making a pull request against the master branch (left side). Also, you should start your branch off our latest master.
  • [ ] My change requires a change to the documentation.
    • [ ] If you've changed APIs, describe what needs to be updated in the documentation.
    • [ ] If new config option added, ensure that it can be set via ENV variable
  • [ ] I have updated the documentation accordingly.
  • [ ] Modules and vendor dependencies have been updated; run go mod tidy && go mod vendor
  • [ ] When updating library version must provide reason/explanation for this update.
  • [ ] I have added tests to cover my changes.
  • [ ] All new and existing tests passed.
  • [x] Check your code additions will not fail linting checks:
    • [x] go fmt -s
    • [x] go vet
+29 -21

0 comment

5 changed files

sredxny

pr closed time in 8 days

push eventTykTechnologies/tyk-pump

tbuchaillot

commit sha db9823094919e6b5bdee8ad6bf6758587c1f1b31

Fix base64 ElasticSearch decoding (#257)

view details

push time in 8 days

pull request commentTykTechnologies/tyk-pump

[TN-6] Fix base64 ES decoding

/release to release-1.0

tbuchaillot

comment created time in 8 days

push eventTykTechnologies/tyk-pump

tbuchaillot

commit sha bda5b7c3537bb415e97c3247708815779d7b957f

changing healthcheck

view details

tbuchaillot

commit sha 63bc0c51c682808f40675b849641db140877a69d

bluring username and password in ES logs

view details

push time in 8 days

push eventmatiasinsaurralde/tyk-pump

tbuchaillot

commit sha 93c540931d29e67bfbbd31165104e20661d1140d

adding basic auth for ES

view details

tbuchaillot

commit sha e5c199fdcb6f140e9046d9423611af7f798d29d6

adding basic auth for Kafka

view details

tbuchaillot

commit sha bbdbf9747613b355d6fbe753e9eb01ffd62a616e

added logs for kafka and ApiKey auth for ES

view details

tbuchaillot

commit sha 2e772e3b069f8ae167568de1799f2fa9d8bdd546

Merge branch 'master' of github.com:TykTechnologies/tyk-pump into auth-Kafka-ES

view details

tbuchaillot

commit sha 279a502a14c537a329f9d6feac77ba7888bb8ebc

adding auth_api_key_id

view details

tbuchaillot

commit sha 416d1c7f30b176438404c5d59e6f89dfe4f51a44

fixing b64 decoding

view details

tbuchaillot

commit sha 624ad9abbe2ec9f22d42a7ee6f1d795e2cf48c53

fmting

view details

tbuchaillot

commit sha 42200b8feb1377abb1a490b22b40671bd3dce964

bluring username and password in ES logs

view details

tbuchaillot

commit sha 79a3f8f363d777d3bebd2fe3da2667d160d5e1ee

mTLS config for kafka

view details

tbuchaillot

commit sha 238b2f583c6115f6da7cd4aa613bbfc9e56d8856

changing key_file json name

view details

tbuchaillot

commit sha fbb02432a67a29ed84e813e60d519bbff785cfc9

changing healthcheck

view details

tbuchaillot

commit sha f8607018d6157e9bdbb50c72a374c54944f2f947

adding --omit-details options

view details

tbuchaillot

commit sha 4ad7005e47d5f83ba88f205c466ee32b4955d22b

updating gomod requirements

view details

tbuchaillot

commit sha 08c3489c5fa2b3ffdb96c718a7a9bcb4b6fcb8da

adding omit_details configuration

view details

tbuchaillot

commit sha 81f652456d19d2a84e54af3acca1c7a738ef05b6

update readme

view details

tbuchaillot

commit sha 8fafa5681c31c0585463ec883c0d33fab927875e

changed omit_details for omit_detailed_recording

view details

Alok G Singh

commit sha 5835f7b23acda3cf856dcc3e7524917558ed9f8f

Syncing integration workflow from tyk-ci (#273) Quick one-liner to change state name

view details

tbuchaillot

commit sha fc133d3da87c9844e5fab0f3fdcab5869d313c4c

Merge branch 'master' of github.com:TykTechnologies/tyk-pump into feature/ignore-detailed

view details

Vinci Xu

commit sha f56813989e35b71cbfc24836658b9cb2342bddfd

fix: missing comma in kafka config

view details

Matias Insaurralde

commit sha 3cca8bdf92f8fc3a4ddd296883b8b86d42c180d0

Merge pull request #247 from TykTechnologies/auth-Kafka-ES [TN-4] [TN-3] Auth for Kafka and ES

view details

push time in 8 days

pull request commentTykTechnologies/tyk-pump

Added omit_detailed_recording configuration

/release to release-1.0

tbuchaillot

comment created time in 8 days

push eventTykTechnologies/tyk-pump

tbuchaillot

commit sha 416d1c7f30b176438404c5d59e6f89dfe4f51a44

fixing b64 decoding

view details

tbuchaillot

commit sha 624ad9abbe2ec9f22d42a7ee6f1d795e2cf48c53

fmting

view details

Matias Insaurralde

commit sha 0adb849f986a5fde9453a928f378c9688d85e1f7

Merge pull request #257 from TykTechnologies/fix/base64ES [TN-6] Fix base64 ES decoding

view details

push time in 8 days

delete branch TykTechnologies/tyk-pump

delete branch : fix/base64ES

delete time in 8 days

PR merged TykTechnologies/tyk-pump

[TN-6] Fix base64 ES decoding bug

Fixing base64 decoding for ElasticSearch

Description

Fixing the base64 decoding for ElasticSearch raw request and response.

Related Issue

<!-- This project only accepts pull requests related to open issues --> <!-- If suggesting a new feature or change, please discuss it in an issue first --> <!-- If fixing a bug, there should be an issue describing it with steps to reproduce --> <!-- Please link to the issue here --> Issue #52

Motivation and Context

<!-- Why is this change required? What problem does it solve? --> The config option decode_base64 for ElasticSearch wasn't working.

How This Has Been Tested

<!-- Please describe in detail how you tested your changes --> <!-- Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. --> Tested with decode_base64 true and false. It works ok.

Screenshots (if appropriate)

Analytic record with decode_base64false: image Analytic record with decode_base64true: image

Types of changes

<!-- What types of changes does your code introduce? Put an x in all the boxes that apply: -->

  • [X] Bug fix (non-breaking change which fixes an issue)
  • [ ] New feature (non-breaking change which adds functionality)
  • [ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist

<!-- Go over all the following points, and put an x in all the boxes that apply --> <!-- If you're unsure about any of these, don't hesitate to ask; we're here to help! -->

  • [X] Make sure you are requesting to pull a topic/feature/bugfix branch (right side). If pulling from your own fork, don't request your master!
  • [X] Make sure you are making a pull request against the master branch (left side). Also, you should start your branch off our latest master.
  • [] My change requires a change to the documentation.
    • [ ] If you've changed APIs, describe what needs to be updated in the documentation.
  • [ ] I have updated the documentation accordingly.
  • [ ] Modules and vendor dependencies have been updated; run go mod tidy && go mod vendor
  • [ ] I have added tests to cover my changes.
  • [X] All new and existing tests passed.
  • [X] Check your code additions will not fail linting checks:
    • [X] go fmt -s
    • [X] go vet
+4 -2

0 comment

1 changed file

tbuchaillot

pr closed time in 8 days

issue closedTykTechnologies/tyk-pump

Raw data is sent in base64 to Elasticsearch

When you use Which is pretty useless for use with kibana if you want to search in the body of the requests or the answers. There should be some automated based64decode before raw_data is sent to elasticsearch

closed time in 8 days

jmleoni

push eventTykTechnologies/tyk-pump

tbuchaillot

commit sha f8607018d6157e9bdbb50c72a374c54944f2f947

adding --omit-details options

view details

tbuchaillot

commit sha 4ad7005e47d5f83ba88f205c466ee32b4955d22b

updating gomod requirements

view details

tbuchaillot

commit sha 08c3489c5fa2b3ffdb96c718a7a9bcb4b6fcb8da

adding omit_details configuration

view details

tbuchaillot

commit sha 81f652456d19d2a84e54af3acca1c7a738ef05b6

update readme

view details

tbuchaillot

commit sha 8fafa5681c31c0585463ec883c0d33fab927875e

changed omit_details for omit_detailed_recording

view details

tbuchaillot

commit sha fc133d3da87c9844e5fab0f3fdcab5869d313c4c

Merge branch 'master' of github.com:TykTechnologies/tyk-pump into feature/ignore-detailed

view details

Matias Insaurralde

commit sha ae19a7f49593de87d25c2a3ed5278f03858623fe

Merge pull request #269 from TykTechnologies/feature/ignore-detailed Added omit_detailed_recording configuration

view details

push time in 8 days

delete branch TykTechnologies/tyk-pump

delete branch : feature/ignore-detailed

delete time in 8 days

PR merged TykTechnologies/tyk-pump

Reviewers
Added omit_detailed_recording configuration enhancement

<!-- Provide a general summary of your changes in the Title above -->

Description

<!-- Describe your changes in detail --> Fix https://github.com/TykTechnologies/tyk-sink/issues/78

Added omit_detailed_recording config option at the root level and in each pump config to avoid writing raw_request and raw_response fields for each request in pumps.

Root level config has more priority than the pump level config.

Example root level config:

{
  "analytics_storage_config": {
    ....
  },
  "analytics_storage_type": "redis",
  "dont_purge_uptime_data": true,
  "pumps": {
   "csv": {
      "type": "csv",
      "meta": {
        "csv_dir": "csv"
      }
    }
  },
  "purge_delay": 1,
  "omit_detailed_recording": true
}

Example pump level config:

...
  "pumps": {
   "csv": {
      "type": "csv",
      "omit_detailed_recording":true,
      "meta": {
        "csv_dir": "csv"
      }
    }
  },
..
}

Related Issue

<!-- This project only accepts pull requests related to open issues --> <!-- If suggesting a new feature or change, please discuss it in an issue first --> <!-- If fixing a bug, there should be an issue describing it with steps to reproduce --> <!-- Please link to the issue here -->

Motivation and Context

<!-- Why is this change required? What problem does it solve? --> Mostly to avoid cloud issues.

How This Has Been Tested

<!-- Please describe in detail how you tested your changes --> <!-- Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. --> Manually tested with root level omit_detailed_recording config. Added test to validate the omit_detailed_recording at pump level config.

Screenshots (if appropriate)

Types of changes

<!-- What types of changes does your code introduce? Put an x in all the boxes that apply: -->

  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [X] New feature (non-breaking change which adds functionality)
  • [ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist

<!-- Go over all the following points, and put an x in all the boxes that apply --> <!-- If you're unsure about any of these, don't hesitate to ask; we're here to help! -->

  • [X] Make sure you are requesting to pull a topic/feature/bugfix branch (right side). If pulling from your own fork, don't request your master!
  • [X] Make sure you are making a pull request against the master branch (left side). Also, you should start your branch off our latest master.
  • [ ] My change requires a change to the documentation.
    • [ ] If you've changed APIs, describe what needs to be updated in the documentation.
  • [ ] I have updated the documentation accordingly.
  • [x] Modules and vendor dependencies have been updated; run go mod tidy && go mod vendor
  • [ ] I have added tests to cover my changes.
  • [X] All new and existing tests passed.
  • [X] Check your code additions will not fail linting checks:
    • [X] go fmt -s
    • [X] go vet
+110 -318

9 comments

25 changed files

tbuchaillot

pr closed time in 8 days

push eventTykTechnologies/tyk-pump

tbuchaillot

commit sha fbb02432a67a29ed84e813e60d519bbff785cfc9

changing healthcheck

view details

tbuchaillot

commit sha 1ee163cf2b7a98fb3349f722617e9dfa757bf771

master merge

view details

Matias Insaurralde

commit sha 80e2b6b84511816af50bd24150f7a78c903089d1

Merge pull request #260 from TykTechnologies/bugfix/healtcheck-endpoint [TN-8] Healthcheck endpoint issues

view details

push time in 8 days

delete branch TykTechnologies/tyk-pump

delete branch : bugfix/healtcheck-endpoint

delete time in 8 days

PR merged TykTechnologies/tyk-pump

[TN-8] Healthcheck endpoint issues bug

Description

<!-- Describe your changes in detail --> Changing default healthcheck endpoint port to 8083 so it doesn't have a conflict with the gateway. Removing localhost from ListenAndServe so it can work on Docker / k8s. If host is omitted it listen on all available interfaces instead. Adding an error check on ListenAndServe so if the port is taken, the pump fails to initialize.

I updated the README but it needs to be updated in the documentation: https://github.com/TykTechnologies/tyk-docs/blob/0d05d16c09be9231b0bfa38cabc310b8d4eee6c1/tyk-docs/content/tyk-configuration-reference/tyk-pump-configuration/tyk-pump-configuration.md

Related Issue

<!-- This project only accepts pull requests related to open issues --> <!-- If suggesting a new feature or change, please discuss it in an issue first --> <!-- If fixing a bug, there should be an issue describing it with steps to reproduce --> <!-- Please link to the issue here --> Closes #256

How This Has Been Tested

<!-- Please describe in detail how you tested your changes --> <!-- Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. --> Locally - Docker.

Screenshots (if appropriate)

Types of changes

<!-- What types of changes does your code introduce? Put an x in all the boxes that apply: -->

  • [x] Bug fix (non-breaking change which fixes an issue)
  • [ ] New feature (non-breaking change which adds functionality)
  • [ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist

<!-- Go over all the following points, and put an x in all the boxes that apply --> <!-- If you're unsure about any of these, don't hesitate to ask; we're here to help! -->

  • [x] Make sure you are requesting to pull a topic/feature/bugfix branch (right side). If pulling from your own fork, don't request your master!
  • [x] Make sure you are making a pull request against the master branch (left side). Also, you should start your branch off our latest master.
  • [x] My change requires a change to the documentation.
    • [x] If you've changed APIs, describe what needs to be updated in the documentation.
  • [x] I have updated the documentation accordingly.
  • [ ] Modules and vendor dependencies have been updated; run go mod tidy && go mod vendor
  • [ ] I have added tests to cover my changes.
  • [x] All new and existing tests passed.
  • [x] Check your code additions will not fail linting checks:
    • [x] go fmt -s
    • [x] go vet

Testing instructions

  • Setup a basic Tyk Pump installation.
  • Start Tyk Pump with the default configuration values.
  • Send a request to http://localhost: 8080/hello (default health check endpoint, with default port).
  • Modify tyk_pump.conf with the following setting: "health_check_endpoint_port": 8083.
  • Start Tyk Pump again and send a request to http://localhost: 8083/hello, expecting HTTP 200.
+9 -5

3 comments

2 changed files

tbuchaillot

pr closed time in 8 days

push eventTykTechnologies/tyk-pump

tbuchaillot

commit sha 42200b8feb1377abb1a490b22b40671bd3dce964

bluring username and password in ES logs

view details

tbuchaillot

commit sha ecadb0a720d83baad6f76203bfa6c7d74404d0db

master merge

view details

Matias Insaurralde

commit sha da9f02879d8422a4b0da2a96ba94a7abfc5e9d06

Merge pull request #258 from TykTechnologies/bugfix/es-url-print [TN-7] Bluring username and password in ES pump logs

view details

push time in 8 days

delete branch TykTechnologies/tyk-pump

delete branch : bugfix/es-url-print

delete time in 8 days

PR merged TykTechnologies/tyk-pump

[TN-7] Bluring username and password in ES pump logs bug

Description

<!-- Describe your changes in detail -->The idea behind this PR is to hide the username and password in ES pump logs when the URL has the following format: https://username:password@rgb-d4.blabla.com:9200”

Related Issue

<!-- This project only accepts pull requests related to open issues --> <!-- If suggesting a new feature or change, please discuss it in an issue first --> <!-- If fixing a bug, there should be an issue describing it with steps to reproduce --> <!-- Please link to the issue here --> Issue #240

How This Has Been Tested

<!-- Please describe in detail how you tested your changes --> <!-- Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. --> Tested using a elasticsearch_url with the following format: https://elasticURL:port/ and https://username:password@elasticURL:port/

Screenshots (if appropriate)

elasticsearch_url with admin:password@ image

elasticsearch_url without admin:password@ image

Types of changes

<!-- What types of changes does your code introduce? Put an x in all the boxes that apply: -->

  • [x] Bug fix (non-breaking change which fixes an issue)
  • [ ] New feature (non-breaking change which adds functionality)
  • [ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist

<!-- Go over all the following points, and put an x in all the boxes that apply --> <!-- If you're unsure about any of these, don't hesitate to ask; we're here to help! -->

  • [x] Make sure you are requesting to pull a topic/feature/bugfix branch (right side). If pulling from your own fork, don't request your master!
  • [x] Make sure you are making a pull request against the master branch (left side). Also, you should start your branch off our latest master.
  • [ ] My change requires a change to the documentation.
    • [ ] If you've changed APIs, describe what needs to be updated in the documentation.
  • [ ] I have updated the documentation accordingly.
  • [ ] Modules and vendor dependencies have been updated; run go mod tidy && go mod vendor
  • [ ] I have added tests to cover my changes.
  • [x] All new and existing tests passed.
  • [x] Check your code additions will not fail linting checks:
    • [x] go fmt -s
    • [x] go vet
+5 -1

0 comment

1 changed file

tbuchaillot

pr closed time in 8 days

issue closedTykTechnologies/tyk-pump

Log message shouldn't print the username and password of Elasticsearch

https://github.com/TykTechnologies/tyk-pump/blob/0bd09b3835bd797e5944c713409892fc4be9a95a/pumps/elasticsearch.go#L214

time="Feb 23 13:13:13" level=info msg="Elasticsearch URL: https://admin:password@rgb-d4.blabla.com:9200/”

closed time in 8 days

letzya

push eventTykTechnologies/tyk-pump

tbuchaillot

commit sha 79a3f8f363d777d3bebd2fe3da2667d160d5e1ee

mTLS config for kafka

view details

tbuchaillot

commit sha 238b2f583c6115f6da7cd4aa613bbfc9e56d8856

changing key_file json name

view details

tbuchaillot

commit sha a9b6d7995b076babd27c2da724a3071f42bc429c

master merger

view details

tbuchaillot

commit sha f25a9eb2d94b43b5910c09be27d8fcd4ee29a5c2

update kafka config readme

view details

Matias Insaurralde

commit sha b248b8aed96ac7f57422acf74e01857286e35e15

Merge pull request #259 from TykTechnologies/feature/kafka-mTLS [TN-1] mTLS support for Kafka

view details

push time in 10 days

delete branch TykTechnologies/tyk-pump

delete branch : feature/kafka-mTLS

delete time in 10 days

PR merged TykTechnologies/tyk-pump

Reviewers
[TN-1] mTLS support for Kafka enhancement

<!-- Provide a general summary of your changes in the Title above --> Added mTLS support for Kafka pump.

Description

<!-- Describe your changes in detail --> Added mTLS support for Kafka pump. It is enabled if the config option use_ssl is set to true and both ssl_cert_file and ssl_key_file are settled. If only one of ssl_cert_file and ssl_key_file config option is set, pump is going to show an error log. If none of these options are settled, it's going to try without it.

Example of Kafka configuration:

    "kafka": {
      "type": "kafka",
      "meta": {
        "broker": [
            "pkc-4yyd6.us-east1.gcp.confluent.cloud:9092"
        ],
        "client_id": "lkc-9wmd7",
        "topic": "test-topic",
        "timeout": 60,
        "compressed": true,
        "meta_data": {
            "key": "value"
        },
        "ssl_insecure_skip_verify":true,
        "use_ssl":true,
        "ssl_cert_file":"<cert-path>",
        "ssl_key_file":"<key-path>",
      }
    }

Related Issue

<!-- This project only accepts pull requests related to open issues --> <!-- If suggesting a new feature or change, please discuss it in an issue first --> <!-- If fixing a bug, there should be an issue describing it with steps to reproduce --> <!-- Please link to the issue here --> Closes Issue #235

Motivation and Context

<!-- Why is this change required? What problem does it solve? -->

How This Has Been Tested

<!-- Please describe in detail how you tested your changes --> <!-- Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. -->

  1. Clone https://github.com/tbuchaillot/kafka-mtls
  2. cd kafka-mtls and execute ./start.sh
  3. Copy the output data, for example: image
  4. Configure pump with that data, with the data of the example above, kafka pump should look like:
"kafka": {
      "type": "kafka",
      "meta": {
        "broker": [
            "localhost:9092"
        ],
        "client_id": "1",
        "topic": "test",
        "timeout": 60,
        "compressed": true,
        "meta_data": {
            "key": "value"
        },
        "ssl_insecure_skip_verify":true,
        "use_ssl":true,        
        "ssl_cert_file":"/Users/tomuz/go/src/github.com/tbuchaillot/kafka-mtls/certs/client.certificate.pem",
        "ssl_key_file":"/Users/tomuz/go/src/github.com/tbuchaillot/kafka-mtls/certs/client.key"
      }
    }
  1. Execute pump, write a new analytic record.
  2. In the write of the first record, it's going to fail because the kafka cluster is looking for leadership election.
  3. After that, everything have to work fine. You should see INFO kafka-pump: Writing 1 records... in the pump logs.

Screenshots (if appropriate)

Types of changes

<!-- What types of changes does your code introduce? Put an x in all the boxes that apply: -->

  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [x] New feature (non-breaking change which adds functionality)
  • [ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist

<!-- Go over all the following points, and put an x in all the boxes that apply --> <!-- If you're unsure about any of these, don't hesitate to ask; we're here to help! -->

  • [ ] Make sure you are requesting to pull a topic/feature/bugfix branch (right side). If pulling from your own fork, don't request your master!
  • [ ] Make sure you are making a pull request against the master branch (left side). Also, you should start your branch off our latest master.
  • [ ] My change requires a change to the documentation.
    • [ ] If you've changed APIs, describe what needs to be updated in the documentation.
  • [ ] I have updated the documentation accordingly.
  • [ ] Modules and vendor dependencies have been updated; run go mod tidy && go mod vendor
  • [ ] I have added tests to cover my changes.
  • [x] All new and existing tests passed.
  • [x] Check your code additions will not fail linting checks:
    • [ ] go fmt -s
    • [ ] go vet
+23 -2

0 comment

2 changed files

tbuchaillot

pr closed time in 10 days

issue closedTykTechnologies/tyk-pump

Add mutual TLS support for Kafka

Is your feature request related to a problem? Please describe. Kafka requires the client to authenticate with a key and certificate, unless you disable that option. We don't currently support this in pump.

Describe the solution you'd like Add mutual TLS support for Kakfa, similar to what is available for Splunk

closed time in 10 days

n2linux

issue commentTykTechnologies/tyk

SSE stream fails to upgrade properly

After more testing, I've found the following, in this particular scenario the issue seems to be caused by enable_detailed_recording, the following definition seems to work fine:

{
    "name": "Tyk Test API",
    "api_id": "3",
    "org_id": "default",
    "definition": {
        "location": "",
        "key": ""
    },
    
    "use_keyless": true,
    "auth": {
        "auth_header_name": ""
    },
    "enable_detailed_recording": false,
    "version_data": {
        "not_versioned": true,
        "versions": {
            "Default": {
                "name": "Default",
                "expires": "3000-01-02 15:04",
                "use_extended_paths": true,
                "extended_paths": {
                    "ignored": [],
                    "white_list": [],
                    "black_list": []
                }
            }
        }
    },
    "proxy": {
        "listen_path": "/quickstart/",
        "target_url": "http://localhost:8000/",
        "strip_listen_path": true
    },
    "do_not_track": true
}

Using the following tyk.conf:

{
    "listen_address": "",
    "listen_port": 8080,
    "secret": "352d20ee67be67f6340b4c0605b044b7",
    "node_secret": "",
    "template_path": "templates",
    "tyk_js_path": "",
    "middleware_path": "middleware",
    "policies": {
        "policy_source": "file",
        "policy_connection_string": "",
        "policy_record_name": "policies/policies.json",
        "allow_explicit_policy_id": false
    },
    "use_db_app_configs": false,
    "db_app_conf_options": {
        "connection_string": "",
        "node_is_segmented": false,
        "tags": null
    },
    "disable_dashboard_zeroconf": false,
    "app_path": "apps/",
    "storage": {
        "type": "redis",
        "host": "localhost",
        "port": 6379,
        "hosts": null,
        "username": "",
        "password": "",
        "database": 0,
        "optimisation_max_idle": 100,
        "optimisation_max_active": 0,
        "enable_cluster": false,
        "use_ssl": false,
        "ssl_insecure_skip_verify": false
    },
    "enable_separate_cache_store": false,
    "cache_storage": {
        "type": "",
        "host": "",
        "port": 0,
        "hosts": null,
        "username": "",
        "password": "",
        "database": 0,
        "optimisation_max_idle": 0,
        "optimisation_max_active": 0,
        "enable_cluster": false,
        "use_ssl": false,
        "ssl_insecure_skip_verify": false
    },
    "enable_analytics": false,
    "analytics_config": {
        "type": "",
        "ignored_ips": [],
        "enable_detailed_recording": false,
        "enable_geo_ip": false,
        "geo_ip_db_path": "",
        "normalise_urls": {
            "enabled": false,
            "normalise_uuids": false,
            "normalise_numbers": false,
            "custom_patterns": null
        },
        "pool_size": 0,
        "records_buffer_size": 0,
        "storage_expiration_time": 0
    },
    "health_check": {
        "enable_health_checks": false,
        "health_check_value_timeouts": 0
    },
    "optimisations_use_async_session_write": false,
    "session_update_pool_size": 0,
    "session_update_buffer_size": 0,
    "allow_master_keys": true,
    "hash_keys": false,
    "hash_key_function": "",
    "suppress_redis_signal_reload": false,
    "suppress_default_org_store": false,
    "use_redis_log": false,
    "sentry_code": "",
    "use_sentry": false,
    "use_syslog": false,
    "use_graylog": false,
    "use_logstash": false,
    "graylog_network_addr": "",
    "logstash_network_addr": "",
    "syslog_transport": "",
    "logstash_transport": "",
    "syslog_network_addr": "",
    "statsd_connection_string": "",
    "statsd_prefix": "",
    "enforce_org_data_age": false,
    "enforce_org_data_detail_logging": false,
    "enforce_org_quotas": false,
    "experimental_process_org_off_thread": false,
    "enable_non_transactional_rate_limiter": false,
    "enable_sentinel_rate_limiter": false,
    "enable_redis_rolling_limiter": false,
    "management_node": false,
    "monitor": {
        "enable_trigger_monitors": false,
        "configuration": {
            "method": "",
            "target_path": "",
            "template_path": "",
            "header_map": null,
            "event_timeout": 0
        },
        "global_trigger_limit": 0,
        "monitor_user_keys": false,
        "monitor_org_keys": false
    },
    "oauth_refresh_token_expire": 0,
    "oauth_token_expire": 0,
    "oauth_token_expired_retain_period": 0,
    "oauth_redirect_uri_separator": "",
    "slave_options": {
        "use_rpc": false,
        "use_ssl": false,
        "ssl_insecure_skip_verify": false,
        "connection_string": "",
        "rpc_key": "",
        "api_key": "",
        "enable_rpc_cache": false,
        "bind_to_slugs": false,
        "disable_keyspace_sync": false,
        "group_id": "",
        "call_timeout": 0,
        "ping_timeout": 0,
        "rpc_pool_size": 0
    },
    "disable_virtual_path_blobs": false,
    "local_session_cache": {
        "disable_cached_session_state": true,
        "cached_session_timeout": 0,
        "cached_session_eviction": 0
    },
    "http_server_options": {
        "override_defaults": false,
        "read_timeout": 0,
        "write_timeout": 0,
        "use_ssl": false,
        "use_ssl_le": false,
        "ssl_insecure_skip_verify": false,
        "enable_websockets": false,
        "certificates": null,
        "ssl_certificates": null,
        "server_name": "",
        "min_version": 0,
        "flush_interval": 0,
        "skip_url_cleaning": false,
        "skip_target_path_escaping": false,
        "ssl_ciphers": null
    },
    "service_discovery": {
        "default_cache_timeout": 0
    },
    "proxy_close_connections": false,
    "close_connections": false,
    "auth_override": {
        "force_auth_provider": false,
        "auth_provider": {
            "name": "",
            "storage_engine": "",
            "meta": null
        },
        "force_session_provider": false,
        "session_provider": {
            "name": "",
            "storage_engine": "",
            "meta": null
        }
    },
    "uptime_tests": {
        "disable": false,
        "config": {
            "failure_trigger_sample_size": 0,
            "time_wait": 0,
            "checker_pool_size": 0,
            "enable_uptime_analytics": false
        }
    },
    "hostname": "",
    "enable_api_segregation": false,
    "control_api_hostname": "",
    "control_api_port": 0,
    "enable_custom_domains": false,
    "enable_jsvm": true,
    "jsvm_timeout": 0,
    "coprocess_options": {
        "enable_coprocess": false
    },
    "hide_generator_header": false,
    "event_handlers": {
        "events": null
    },
    "event_trigers_defunct": null,
    "pid_file_location": "",
    "allow_insecure_configs": false,
    "public_key_path": "",
    "close_idle_connections": false,
    "drl_notification_frequency": 0,
    "global_session_lifetime": 0,
    "force_global_session_lifetime": false,
    "bundle_base_url": "http://localhost/dev/",
    "enable_bundle_downloader": true,
    "allow_remote_config": false,
    "legacy_enable_allowance_countdown": false,
    "max_idle_connections": 0,
    "max_idle_connections_per_host": 0,
    "max_conn_time": 0,
    "reload_wait_time": 0,
    "proxy_ssl_insecure_skip_verify": false,
    "proxy_ssl_min_version": 0,
    "proxy_ssl_ciphers": null,
    "proxy_default_timeout": 10,
    "log_level": "",
    "security": {
        "private_certificate_encoding_secret": "",
        "control_api_use_mutual_tls": false,
        "pinned_public_keys": null,
        "certificates": {
            "apis": null,
            "upstream": null,
            "control_api": null,
            "dashboard_api": null,
            "mdcb_api": null
        }
    },
    "enable_key_logging": false,
    "newrelic": {
        "app_name": "",
        "license_key": ""
    },
    "version_header": "",
    "enable_hashed_keys_listing": false,
    "min_token_length": 0,
    "disable_regexp_cache": false,
    "regexp_cache_expire": 0,
    "enable_delta_reload": true
}

I've tested the above scenario together with enable_websockets set to false and flush_interval with a lower value like 1 (but not zero!), changing any of these values will produce issues. We should evaluate the following ideas:

  1. Support detailed recording when using SSE.
  2. Support SSE when enable_websockets is set to true.
  3. Change the flush behavior when SSE is in use. In general, being able to detect whether we're using SSE or not should be useful.
jlucktay

comment created time in 10 days

push eventmatiasinsaurralde/tyk

Sredny M

commit sha 110cdf7e9bda403ced957f32b37ccde361c3087e

Fix token update generate new hash entry (#3229) <!-- Provide a general summary of your changes in the Title above --> ## Description On update keys, we should not call twice `doAddOrUpdate` as this create duplicates for the same key in redis, instead, we should check if we are dealing with a custom key or not, and therefore perform the proper action. ## Related Issue https://github.com/TykTechnologies/tyk/issues/3109 ## Motivation and Context Give solution to https://github.com/TykTechnologies/tyk/issues/3109 ## How This Has Been Tested - Run GW and Dashboard with `"hash_keys": true` - Create a custom key via gw api - Update custom key...and check that the value was updated and an additional registry was not created in redis - Create key via dashboard - Update key via dashboard, check that the value was updated but an additional registry in redis was not created for the same key - Do the same but now with `"hash_keys": false` ## Screenshots (if appropriate) ## Types of changes <!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: --> - [x] Bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Refactoring or add test (improvements in base code or adds test coverage to functionality) ## Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply --> <!-- If you're unsure about any of these, don't hesitate to ask; we're here to help! --> - [x] Make sure you are requesting to **pull a topic/feature/bugfix branch** (right side). If pulling from your own fork, don't request your `master`! - [x] Make sure you are making a pull request against the **`master` branch** (left side). Also, you should start *your branch* off *our latest `master`*. - [ ] My change requires a change to the documentation. - [ ] If you've changed APIs, describe what needs to be updated in the documentation. - [ ] If new config option added, ensure that it can be set via ENV variable - [ ] I have updated the documentation accordingly. - [ ] Modules and vendor dependencies have been updated; run `go mod tidy && go mod vendor` - [ ] When updating library version must provide reason/explanation for this update. - [x] I have added tests to cover my changes. - [x] All new and existing tests passed. - [x] Check your code additions will not fail linting checks: - [x] `go fmt -s` - [x] `go vet`

view details

Furkan Senharputlu

commit sha 1cc358813e8fc4f5999537efe75c3d0f7a5364e2

Fix panic when path is empty (#3243) Fixes https://github.com/TykTechnologies/tyk/issues/3242

view details

Furkan Senharputlu

commit sha ff415a6e40e3a31f78d01db9c16df7d537680597

Intersect restricted types when two policies are applied on same key (#3226)

view details

Sergey Petrunin

commit sha 1abd35f707cf66067117f502eb2dedcbea184366

Feature/use debuggable datasources (#3234) * tmp: vendor local graphql-go-tools * add datasource hooks to graphql middleware * use graphql hooks for loaded apis * add upstream url to pre send datasource hook * fix vendoring of graphql-go-tools * add more fields to postReceive hook add type and field name to the log message

view details

Furkan Senharputlu

commit sha 931118f73c0c8f88af4d02345382bc49364a098d

Vendor latest library changes (#3254) Related to #3235

view details

push time in 10 days

push eventTykTechnologies/tyk-pump

Vinci Xu

commit sha f56813989e35b71cbfc24836658b9cb2342bddfd

fix: missing comma in kafka config

view details

Matias Insaurralde

commit sha 69a08bf648ffeb958df35f4a18c45e06e428874b

Merge pull request #276 from ShiningRush/master fix: missing comma in kafka config

view details

push time in 15 days

PR merged TykTechnologies/tyk-pump

fix: missing comma in kafka config

Description

missing comma at kafka section in demo config.

+1 -1

0 comment

1 changed file

ShiningRush

pr closed time in 15 days

push eventTykTechnologies/tyk-pump

tbuchaillot

commit sha 93c540931d29e67bfbbd31165104e20661d1140d

adding basic auth for ES

view details

tbuchaillot

commit sha e5c199fdcb6f140e9046d9423611af7f798d29d6

adding basic auth for Kafka

view details

tbuchaillot

commit sha bbdbf9747613b355d6fbe753e9eb01ffd62a616e

added logs for kafka and ApiKey auth for ES

view details

tbuchaillot

commit sha 2e772e3b069f8ae167568de1799f2fa9d8bdd546

Merge branch 'master' of github.com:TykTechnologies/tyk-pump into auth-Kafka-ES

view details

tbuchaillot

commit sha 279a502a14c537a329f9d6feac77ba7888bb8ebc

adding auth_api_key_id

view details

Matias Insaurralde

commit sha 3cca8bdf92f8fc3a4ddd296883b8b86d42c180d0

Merge pull request #247 from TykTechnologies/auth-Kafka-ES [TN-4] [TN-3] Auth for Kafka and ES

view details

push time in 15 days

delete branch TykTechnologies/tyk-pump

delete branch : auth-Kafka-ES

delete time in 15 days

PR merged TykTechnologies/tyk-pump

Reviewers
[TN-4] [TN-3] Auth for Kafka and ES enhancement

Description

Adding configuration for auth mechanism in Kafka and ElasticSearch:

  • basic auth (plain and scram) for Kafka
  • basic auth (username and password) and API Key auth (api_key and api_key_id) for ElasticSearch.

ElasticSearch: Basic auth configuration: The fields auth_basic_username and auth_basic_password are added into ES configuration. Those fields are going to be send in the Authorization header as username:password encoded in base64. Example config:

"elasticsearch": {
      "type": "elasticsearch",
      "meta": {
        "index_name": "tyk_analytics",
        "elasticsearch_url": "https://elastichost:9243",
        "enable_sniffing": false,
        "document_type": "tyk_analytics",
        "rolling_index": false,
        "extended_stats": false,
        "version": "6",
        "auth_basic_username":"elastic",
        "auth_basic_password":"password"
      }
    }

API Key auth configuration: The fields auth_api_key_id and auth_api_key are added into ES configuration. Those fields are going to be send in the Authorization header as ApiKey base64(auth_api_key_id:auth_api_key). Example config:

"elasticsearch": {
      "type": "elasticsearch",
      "meta": {
        "index_name": "tyk_analytics",
        "elasticsearch_url": "https://elastichost:9243",
        "enable_sniffing": false,
        "document_type": "tyk_analytics",
        "rolling_index": false,
        "extended_stats": false,
        "version": "6",
        "auth_api_key_id":"8bXvD3IB-IBDfS7oZEWV",
        "auth_api_key":"LRmuicydSW-zeb0Zd_3DGw"      
      }
    }

Kafka: The fields sasl_mechanism, sasl_algorithm, sasl_username and sasl_password are added into Kafka configuration.

  • sasl_mechanism: It's the authentication mechanism. It could be plain or scram.
  • sasl_algorithm: It's the algorithm specified for scram mechanism. It could be sha-512 or sha-256, sha-256 it's the default value if the algorithm is not configured.
  • sasl_username: It's the username or token / key.
  • sasl_password: It's the password or secret.

Example config for plain mechanism:

    "kafka": {
      "type": "kafka",
      "meta": {
        "broker": [
            "pkc-4yyd6.us-east1.gcp.confluent.cloud:9092"
        ],
        "client_id": "lkc-9wmd7",
        "topic": "test-topic",
        "timeout": 60,
        "compressed": true,
        "meta_data": {
            "key": "value"
        },
        "use_ssl":true,
        "sasl_mechanism":"plain",
        "sasl_username":"usernameD7SWRYRIQEFEXHNB",
        "sasl_password":"passwordD7SWRYRIQEFEXHNB"
      }
    }

Example config for scram mechanism:

    "kafka": {
      "type": "kafka",
      "meta": {
        "broker": [
            "testtest.confluent.cloud:9092"
        ],
        "client_id": "test-9wmd7",
        "topic": "test-topic",
        "timeout": 60,
        "compressed": true,
        "meta_data": {
            "key": "value"
        },
        "use_ssl":true,
        "sasl_mechanism":"scram",
        "sasl_algorithm":"sha-512",
        "sasl_username":"usernameD7SWRYRIQEFEXHNB",
        "sasl_password":"passwordD7SWRYRIQEFEXHNB"
      }
    }

Related Issue

Issue #246

Motivation and Context

Issue #246. Enterprise clients / SaaS.

How This Has Been Tested

ElasticSearch:

  • basic auth: Tested with https://cloud.elastic.co/home and ES 5 / 6.
  • API Key auth: Test with https://cloud.elastic.co/home and ES 5 / 6 generating the API Key following https://www.elastic.co/guide/en/elasticsearch/reference/6.8/security-api-create-api-key.html.

Kafka: Tested with Kafka cluster in https://confluent.cloud/.

Types of changes

  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [X] New feature (non-breaking change which adds functionality)
  • [ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist

<!-- Go over all the following points, and put an x in all the boxes that apply --> <!-- If you're unsure about any of these, don't hesitate to ask; we're here to help! -->

  • [ ] Make sure you are requesting to pull a topic/feature/bugfix branch (right side). If pulling from your own fork, don't request your master!
  • [X] Make sure you are making a pull request against the master branch (left side). Also, you should start your branch off our latest master.
  • [x] My change requires a change to the documentation.
  • [ ] If you've changed APIs, describe what needs to be updated in the documentation.
  • [ ] I have updated the documentation accordingly.
  • [ ] Modules and vendor dependencies have been updated; run go mod tidy && go mod vendor
  • [ ] I have added tests to cover my changes.
  • [ ] All new and existing tests passed.
  • [X] Check your code additions will not fail linting checks:
    • [X] go fmt -s
    • [X] go vet
+99 -34

1 comment

2 changed files

tbuchaillot

pr closed time in 15 days

issue closedTykTechnologies/tyk-pump

ElasticSearch missing properties for HTTP Authorization Header & Kafka missing properties for SASL JAAS (username + password) and others

Hey guys,

Usually tools like ES require authentication specially when using SaaS it is possible to pass those Auth Tokens (-H "Authorization: ApiKey $ECE_API_KEY") in the configuration? With this option it is not possible to pass the info like: http://user:pass@host... because it needs to be an HTTP header.

DOCS: https://www.elastic.co/guide/en/cloud-enterprise/current/Authentication.html#login

The same for Kafka some SaaS Kafka give you this data:

bootstrap.servers=hostname:9093

sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="token" password="api-key-secret"

sasl.mechanism=PLAIN

security.protocol=SASL_SSL

ssl.protocol=TLSv1.2

Can those be configured somehow via pump.conf ?

FYI @letzya @buger @asoorm

closed time in 15 days

bmonteiro

push eventTykTechnologies/tyk-sync

Alok G Singh

commit sha 5484a20e376986088906adfc380a2aab7c31c330

Do NOT publish release to github. Added a status badge.

view details

Matias Insaurralde

commit sha f7132248338f38023af53f2a45ef6c1cafce7845

Merge pull request #38 from TykTechnologies/goreleaser/init Do NOT publish release to github.

view details

push time in 16 days

delete branch TykTechnologies/tyk-sync

delete branch : goreleaser/init

delete time in 16 days

PR merged TykTechnologies/tyk-sync

Do NOT publish release to github.

Added a status badge.

+3 -1

0 comment

2 changed files

alephnull

pr closed time in 16 days

pull request commentTykTechnologies/tyk-identity-broker

Remove debugging print

In the latest master, this is already handled with log.Debug.

joshblakeley

comment created time in 18 days

push eventTykTechnologies/tyk-identity-broker

Yaara

commit sha 4649a4551dcc772c3aeaab7196ed311fb6ef4719

clarification per client's request 1. Added explanation to the debug env var 2. Added explanation to the user-group-id. It wasn't clear which id we ask the users to set as a value

view details

Mark Southee

commit sha be012569c6ebf496fd930e9189e818b020266747

Update README.md Couple of changes

view details

Yaara

commit sha ac877b43d49cc9ef6927c9819ea9b184556d67db

fixed typos

view details

Yaara

commit sha 6683bebffa8021c4ec88323b82fe5ef3271831a0

fixed typos

view details

Yaara

commit sha ba7f05c4d6cadfd493223d04fd0006fb6c8d4499

typo "and" was removed

view details

Matias Insaurralde

commit sha 34d1bf51c9ca6ab70a6785bc43d7196f22642b55

Merge pull request #105 from TykTechnologies/letzya-patch-1 clarification per client's request

view details

push time in 18 days

push eventTykTechnologies/tyk-identity-broker

sredny buitrago

commit sha 2c1f083c6f34fae0512bfa2d031da183a25bcdaa

added test for email to use on SSO

view details

sredny buitrago

commit sha 64601f484a5abca3cda1be4c6bb983ce104e8db2

added test for CustomGroupId

view details

Matias Insaurralde

commit sha 1ad2b6b69c807c49ad8381028ac325531a0a1aef

Merge pull request #115 from TykTechnologies/test-custom-fields Test custom fields

view details

push time in 18 days

PR merged TykTechnologies/tyk-identity-broker

Reviewers
Test custom fields

<!-- Provide a general summary of your changes in the Title above -->

Description

Added unit tests for: CustomEmailField, CustomUserIDField, CustomGroupIdField. Was done a small refactor of this logic, in order that is easier to test

Related Issue

None, just add test coverage

Motivation and Context

Provide unit test to this section of the profiles

How This Has Been Tested

go run -v -run TestGetEmail
go run -v -run TestGetUserID
go run -v -runTestGetGroupId

Screenshots (if appropriate)

Types of changes

<!-- What types of changes does your code introduce? Put an x in all the boxes that apply: -->

  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [ ] New feature (non-breaking change which adds functionality)
  • [ ] Breaking change (fix or feature that would cause existing functionality to change)
  • [x] Refactoring or add test (improvements in base code or adds test coverage to functionality)

Checklist

<!-- Go over all the following points, and put an x in all the boxes that apply --> <!-- If you're unsure about any of these, don't hesitate to ask; we're here to help! -->

  • [x] Make sure you are requesting to pull a topic/feature/bugfix branch (right side). If pulling from your own fork, don't request your master!
  • [x] Make sure you are making a pull request against the master branch (left side). Also, you should start your branch off our latest master.
  • [ ] My change requires a change to the documentation.
    • [ ] If you've changed APIs, describe what needs to be updated in the documentation.
    • [ ] If new config option added, ensure that it can be set via ENV variable
  • [ ] I have updated the documentation accordingly.
  • [ ] Modules and vendor dependencies have been updated; run go mod tidy && go mod vendor
  • [ ] When updating library version must provide reason/explanation for this update.
  • [x] I have added tests to cover my changes.
  • [x] All new and existing tests passed.
  • [x] Check your code additions will not fail linting checks:
    • [ ] go fmt -s
    • [ ] go vet
+241 -32

0 comment

2 changed files

sredxny

pr closed time in 18 days

push eventTykTechnologies/tyk-identity-broker

sredny buitrago

commit sha f5e9f99a71600c19b04a6560f1371e4a5e38f448

downgraded redis version, added support for sslSkipVerify

view details

sredny buitrago

commit sha b35a77f0698b0281bcd2030849487a0a30d70e58

fix conflicts

view details

sredny buitrago

commit sha 7c87cb09779238bb7d5b3560e89a86170571ffdd

fix conflicts

view details

sredny buitrago

commit sha 361713326a08bb0d0e60ad01b113e8b6b010bec9

added error validation on cert load

view details

sredny buitrago

commit sha 7ba3605cefe7db538aaf82bef597930f3b4312ba

use mainLogger instead log

view details

Matias Insaurralde

commit sha 13c741a13aa18f9bbebadfedb527048fa1889cd4

Merge pull request #112 from TykTechnologies/fix-102-add-SSLSkipVerify-option downgraded redis version, added support for sslSkipVerify

view details

push time in 18 days

PR merged TykTechnologies/tyk-identity-broker

Reviewers
downgraded redis version, added support for sslSkipVerify

Fixes https://github.com/TykTechnologies/tyk-identity-broker/issues/102

Was added a new config to enable/disable SSLInsecureSkipVerify for the HTTP server under HttpServerOptions.SSLInsecureSkipVerify

+512450 -12

3 comments

1159 changed files

sredxny

pr closed time in 18 days

issue closedTykTechnologies/tyk-identity-broker

Add a way to disable TLS verification for the dashboard communication

At the moment TIB does not work with a self signed certificate, if they are not added to OS trusted storage. It complicates PoC environment creation.

closed time in 18 days

buger

push eventTykTechnologies/tyk-identity-broker

joshblakeley

commit sha c42db1e84f6e0ebffe80ad5f74cfde9560b829a1

SAML provider for TIB

view details

joshblakeley

commit sha cd6b9304620cc69ef494b73d92457289d8ca91e7

extend tap interface to handle metadata endpoint for saml - logger cleanup etc

view details

joshblakeley

commit sha 437eb19bc6da357d878d40b20503197517870233

SAMLProvider constant and add configurable name and email claims

view details

joshblakeley

commit sha ada8a80ae8e254bb160fc18beccd02e44774c78d

error handling and failure redirects in place

view details

joshblakeley

commit sha 439674ec7b08b69f30139ee2548ba783cc065c88

change adlogger to samllogger

view details

bmonteiro

commit sha d0defea4d94b144e1c797ce40c14773c67747966

Adding SAML role support for Dashboard Users Groups. Example: profiles.json ... "CustomUserGroupField": "http://schemas.microsoft.com/ws/2008/06/identity/claims/role", "UserGroupMapping": { "7149b3c0-5e5d-48b3-a42a-c73ded0bc232": "5f034003ed3dc300012e48a0", "ab776d34-1d4c-4ee0-9f4b-9a9cae029f4e": "5f033fa1ed3dc300012e489f" } ...

view details

Josh Blakeley

commit sha c5e6fdcf506132097d9234924c3eafea907b0f81

Merge pull request #106 from bmonteiro/saml-provider Adding SAML role support for Dashboard Users Groups. Example:

view details

Matias Insaurralde

commit sha f262d8c6bd303eb2a2f5157fc01988185d35ada9

Merge pull request #103 from TykTechnologies/saml-provider SAML provider for TIB

view details

push time in 18 days

PR merged TykTechnologies/tyk-identity-broker

Reviewers
SAML provider for TIB documentation needed

Fixes https://github.com/TykTechnologies/tyk-identity-broker/issues/7

Docs PR here: https://github.com/TykTechnologies/tyk-docs/pull/1256

  • Tested with AzureAD and samltest.id

  • [x] Redirect

  • [x] POST

  • [x] Claim mapping

  • [x] docs

Docs:

SSO with SAML and Tyk

SAML authentication is a way for a service provider, such as the Tyk Dashboard or Portal, to assert the Identity of a User via a third party.

Tyk Identity Broker can act as the go-between for the Tyk Dashboard and Portal and a third party identity provider. Tyk Identity broker can also interpret and pass along information about the user who is logging in such as Name, Email and group or role metadata for enforcing role based access control in the Tyk Dashboard.

The provider config for SAML has the following values that can be configured in a Profile:

SAMLBaseURL - The host of TIB that will be used in the metadata document for the Service Provider. This will form part of the metadata URL used as the Entity ID by the IDP. The redirects configured in the IDP must match the expected Host and URI configured in the metadata document made available by Tyk Identity Broker.

FailureRedirect - Where to redirect failed login requests.

IDPMetaDataURL - The metadata URL of your IDP which will provide Tyk Identity Broker with information about the IDP such as EntityID, Endpoints (Single Sign On Service Endpoint, Single Logout Service Endpoint), its public X.509 cert, NameId Format, Organization info and Contact info.

This metadata XML can be signed providing a public X.509 cert and the private key.

CertFile - An X.509 certificate for signing your requests to the IDP

'KeyFile' - A private key for signing your requests to the IDP

ForceAuthentication - Ignore any session held by the IDP and force re-login every request.

SAMLEmailClaim - Key for looking up the email claim in the SAML assertion form the IDP. Defaults to: http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress

SAMLForenameClaim - Key for looking up the forename claim in the SAML assertion form the IDP. Defaults to: http://schemas.xmlsoap.org/ws/2005/05/identity/claims/forename

SAMLSurnameClaim - Key for looking up the surname claim in the SAML assertion form the IDP. Defaults to: http://schemas.xmlsoap.org/ws/2005/05/identity/claims/surname

Example profile configuration:

{
    "ActionType": "GenerateOrLoginUserProfile",
    "ID": "saml-sso-login",
    "OrgID": "{YOUR_ORGANISATION_ID}",
    "CustomEmailField": "",
    "IdentityHandlerConfig": {
        "DashboardCredential": "{DASHBOARD_USER_API_KEY}"
    },
    "ProviderConfig": {
        "SAMLBaseURL": "https://{HOST}",
        "FailureRedirect": "http://{DASHBOARD_HOST}:{PORT}/?fail=true",
        "IDPMetaDataURL": "{IDP_METADATA_URL}",
        "CertFile":"myservice.cert",
        "KeyFile": "myservice.key",
        "ForceAuthentication": false,
        "SAMLEmailClaim": "",
        "SAMLForenameClaim": "",
        "SAMLSurnameClaim": ""
    },
    "ProviderName": "SAMLProvider",
    "ReturnURL": "http://{DASHBOARD_URL}:{PORT}/tap",
    "Type": "redirect"
}
+368 -31

6 comments

10 changed files

joshblakeley

pr closed time in 18 days

issue closedTykTechnologies/tyk-identity-broker

Add an Identity Provider for SAML

TIB should be able to validate SAML authentication to provide access to things such as the Portal and the Dashboard

closed time in 18 days

lonelycode

Pull request review commentTykTechnologies/tyk-identity-broker

downgraded redis version, added support for sslSkipVerify

 func main() {  	p.Handle("/health", http.HandlerFunc(HandleHealthCheck)).Methods("GET") -	listenPort := "3010"+	listenPort := 3010 	if config.Port != 0 {-		listenPort = strconv.Itoa(config.Port)+		listenPort = config.Port 	} -	if config.HttpServerOptions.UseSSL {-		mainLogger.Info("Broker Listening on SSL:", listenPort)-		err := http.ListenAndServeTLS(":"+listenPort, config.HttpServerOptions.CertFile, config.HttpServerOptions.KeyFile, p)-		if err != nil {-			mainLogger.Fatal("ListenAndServe: ", err)+	var tibServer net.Listener+	if config.HttpServerOptions.UseSSL{+		log.Info("--> Using SSL (https) for TIB")+		cert, _:= tls.LoadX509KeyPair(config.HttpServerOptions.CertFile, config.HttpServerOptions.KeyFile)

Could be useful to cover errors here

sredxny

comment created time in 22 days

push eventTykTechnologies/tyk-identity-broker

sredny buitrago

commit sha cf436133e1180bb09bb462e9c071ca078958faf0

migrated to go modules,vendorized and downgraded go-redis version to be fully compatible with dashboard

view details

Matias Insaurralde

commit sha b711d811c2425f540b8631a04ecc4ff405c81f03

Merge pull request #108 from TykTechnologies/revendor-and-migrate-to-go-modules migrated to go modules,vendorized and downgraded go-redis version to be fully compatible with dashboard

view details

push time in 22 days

PR merged TykTechnologies/tyk-identity-broker

Reviewers
migrated to go modules,vendorized and downgraded go-redis version to be fully compatible with dashboard

In this PR 3 things were done:

  • Moved to go modules
  • Vendorized dependencies using go mod vendor
  • Downgraded version of go-redis, because we were using go-redisV8 but it's in beta version and makes kind of messy the vendorization process. Currently dashboard uses v6.15.6 so, in order to use go modules in TIB then we would import the library as import github.com/go-redis/redis/v8 and this will break the logic of tib embeded in dashboard, so, we should use the same version in both applications.
+133724 -114956

0 comment

672 changed files

sredxny

pr closed time in 22 days

PR opened TykTechnologies/tyk

Fix JSVM session metadata usage

Fix for #3218.

Description

When use_session is enabled, the SessionMeta object is expected to contain a content with the following structure:

{"tyk_developer_id":"5f1937ad4ea4611eefb8ede6","tyk_key_request_fields":{},"tyk_user_fields":{}}

Because we were previously using map[string]string, the key request and user fields break the unmarshalling step. Using map[string]interface{} fixes the issue.

Related Issue

#3218.

How This Has Been Tested

Steps described in #3218.

Screenshots (if appropriate)

Types of changes

<!-- What types of changes does your code introduce? Put an x in all the boxes that apply: -->

  • [x] Bug fix (non-breaking change which fixes an issue)
  • [ ] New feature (non-breaking change which adds functionality)
  • [ ] Breaking change (fix or feature that would cause existing functionality to change)
  • [ ] Refactoring or add test (improvements in base code or adds test coverage to functionality)

Checklist

<!-- Go over all the following points, and put an x in all the boxes that apply --> <!-- If you're unsure about any of these, don't hesitate to ask; we're here to help! -->

  • [x] Make sure you are requesting to pull a topic/feature/bugfix branch (right side). If pulling from your own fork, don't request your master!
  • [x] Make sure you are making a pull request against the master branch (left side). Also, you should start your branch off our latest master.
  • [ ] My change requires a change to the documentation.
    • [ ] If you've changed APIs, describe what needs to be updated in the documentation.
    • [ ] If new config option added, ensure that it can be set via ENV variable
  • [ ] I have updated the documentation accordingly.
  • [ ] Modules and vendor dependencies have been updated; run go mod tidy && go mod vendor
  • [ ] When updating library version must provide reason/explanation for this update.
  • [ ] I have added tests to cover my changes.
  • [x] All new and existing tests passed.
  • [ ] Check your code additions will not fail linting checks:
    • [ ] go fmt -s
    • [ ] go vet
+2 -2

0 comment

1 changed file

pr created time in 23 days

create barnchmatiasinsaurralde/tyk

branch : virtual-endpoint-session-metadata

created branch time in 23 days

push eventmatiasinsaurralde/tyk

Patric Vormstein

commit sha 7c48b5b185a2f38d747bd321b083643fa866f6f1

add last_schema_update field to graphql api definition (#3210) Introduce `last_schema_update` field to api definition. This is an optional field that will be used for proxyOnly mode. Related to https://github.com/TykTechnologies/product/issues/361

view details

Alok G Singh

commit sha e5d8700b7ea39989f98f3797b75e47bb5fb70583

Reworking terraform state (#3213) A quick one liner in the workflow to change the name.

view details

Sredny M

commit sha a10b1d2497bd94d935e753e932e8d19056e44b82

Fix custom keys in MDCB (#3129) <!-- Provide a general summary of your changes in the Title above --> ## Description When users attempts to create a custom key, then we should allow them to use this key in any of the slaves nodes. So, at the moment of retriving a key we should handle as well the situation of customs keys, therefore we should check the existence of that key by searching for `generateToken(orgID, keyName) ` In other hand, as we might use a key by alias or by token value then the slave nodes were storing the api based with the value that we used, this is not correct at all as we need to keep consistency in the values stored in redis (to avoid issues on update and delete operations). Basically if we used the alias to generate the hash value (if enabled) and then store that in redis, later if we used the token instead the alias we again hash the value (if enabled) and store that in redis, that made us having duplicates of the same key, in order to fix that then we now consider the option that the key can be modified inside `CheckSessionAndIdentityForValidKey` therefore now that function will receive a `*string` instead `string` (refer to https://github.com/TykTechnologies/tyk/pull/3129/files#diff-76df41fc5dc0183d026b785861779d24R688 ) ## Related Issue https://github.com/TykTechnologies/tyk/issues/3103 ## Motivation and Context Give solution to https://github.com/TykTechnologies/tyk/issues/3103 so custom keys works in MDCB environment. ## How This Has Been Tested 1- Setup MDCB environment 2- Create API and Policy. Set API with authentication token method 2- Create custom key in Master GW...something like: ``` curl --location --request POST 'http://tyk-gateway:8080/tyk/keys/mycustomkey' \ --header 'x-tyk-authorization: 352d20ee67be67f6340b4c0605b044b7' \ --header 'Content-Type: application/json' \ --data-raw '{ "allowance":1000, "rate":1000, "per":60, "quota_max":-1, "quota_renews":1552996743, "quota_remaining":-1, "quota_renewal_rate":-1, "org_id": "5e2091c4d4aefce60c04fb92", "apply_policies":[ "5ed7cb95d4aefc2e388ecb2a" ] }' ``` 3- Consume API in slave node using `mycustomkey` as Authorization 4- You should be allowed to consume API. Eg: ``` curl --location --request GET 'http://tyk-gateway:8182/api-custom-key/' \ --header 'Authorization: mycustomkey' ``` 5- As an extra step, we can check the key details doing something like: ``` curl --location --request GET 'tyk-gateway:8080/tyk/keys/mycustomkey?username=true&org_id=5e2091c4d4aefce60c04fb92' \ --header 'X-Tyk-Authorization: 352d20ee67be67f6340b4c0605b044b7' ``` ## Screenshots (if appropriate) ## Types of changes <!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: --> - [x] Bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) ## Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply --> <!-- If you're unsure about any of these, don't hesitate to ask; we're here to help! --> - [x] Make sure you are requesting to **pull a topic/feature/bugfix branch** (right side). If pulling from your own fork, don't request your `master`! - [x] Make sure you are making a pull request against the **`master` branch** (left side). Also, you should start *your branch* off *our latest `master`*. - [ ] My change requires a change to the documentation. - [ ] If you've changed APIs, describe what needs to be updated in the documentation. - [ ] If new config option added, ensure that it can be set via ENV variable - [ ] I have updated the documentation accordingly. - [ ] Modules and vendor dependencies have been updated; run `go mod tidy && go mod vendor` - [ ] When updating library version must provide reason/explanation for this update. - [ ] I have added tests to cover my changes. - [ ] All new and existing tests passed. - [x] Check your code additions will not fail linting checks: - [x] `go fmt -s` - [x] `go vet`

view details

push time in 23 days

pull request commentTykTechnologies/tyk-pump

Added omit_detailed_recording configuration

@tbuchaillot We have a conflict in pumps/hybrid.go.

tbuchaillot

comment created time in 24 days

PR opened TykTechnologies/tyk

mw_virtual_endpoint: copy HTTP protocol version from request

Improves the analytics record data.

+3 -3

0 comment

2 changed files

pr created time in 25 days

push eventmatiasinsaurralde/tyk

Matias Insaurralde

commit sha d4c7d52c92a09ad9d47cc290b380ff481bbf5e27

mw_virtual_endpoint: copy HTTP protocol version from request

view details

push time in 25 days

create barnchmatiasinsaurralde/tyk

branch : virtual-endpoint-http

created branch time in 25 days

issue commentTykTechnologies/tyk

ID Extractor Plugin Not Working With Multiple Auth

Hi, could you set the gateway log level in debug mode, trigger a request and share the full log? You can achieve this by setting the TYK_LOGLEVEL environment variable to debug or by setting the log_level property to debug in tyk.conf. It would be useful to inspect your plugin output as well (I see that there are some logging instructions in place).

nerdydread

comment created time in 25 days

issue commentTykTechnologies/tyk

SSE stream fails to upgrade properly

enable_websockets set to false and a low flush interval value should be enough for SSE to work. As @bmonteiro mentions, timeouts should use high values or 0. I've tried a similar scenario a few weeks ago.

jlucktay

comment created time in 25 days

issue commentTykTechnologies/tyk

Not able to load multiple native Go plugins

This is fixed by #3196.

matiasinsaurralde

comment created time in 25 days

Pull request review commentTykTechnologies/tyk

Add some documentation and tests for goplugins

+# Tyk Plugin compiler++## Building a plugin+Navigate to where your plugin is and build using a docker volume to

Docker

alephnull

comment created time in 25 days

Pull request review commentTykTechnologies/tyk

Add some documentation and tests for goplugins

+# Tyk Plugin compiler++## Building a plugin+Navigate to where your plugin is and build using a docker volume to+mount your code into the image. Since the vendor directory needs to be+identical between the gateway build and the plugin build, this means+that you should pull the version of this image corresponding to the+gateway version you are using.++This also implies that if your plugin has vendored modules that are+also used by Tyk gateway then your module will be overridden by the+version that Tyk uses.++```shellsession+% docker run -v `pwd`:/plugin-source tykio/tyk-plugin-compiler:v2.9.4.2 myplugin.so+```++You will find a myplugin.so in the current directory which is the file+that goes into the API definition++## Plugin aliasing+Plugins are loaded via `dl_open(3)` and the shared library cache,+`ld.so.cache` will be used. Therefore,even if a plugin's content+changes but the filename does not, the cached plugin will be used.++See the manpages for `dl_open(3)` and `ld.so(8)` on your platform for+more details.++## Hot reloading+Using `/tyk/reload/group` will _not_ update the plugin if you have+compiled a fresh version and used the same pathname.++To reload a plugin, you will have to restart the gateway process.

This is hard to fix unless we implement our own dynamic loading logic. We should wait for the plugin package to implement something that allows the safe unloading/reloading of Go shared libraries, etc. For now the only strategy is to use different filenames every time the code changes or perform a full process reload.

alephnull

comment created time in 25 days

Pull request review commentTykTechnologies/tyk-docs

Removed wrong information

 url: "/plugins/supported-languages/rich-plugins/grpc" aliases:    -  "/plugins/rich-plugins/grpc" ----## What is gRPC?-From the [gRPC documentation](http://www.grpc.io/faq/ ):--> gRPC is a modern, open source remote procedure call (RPC) framework that can run anywhere. It enables client and server applications to communicate transparently, and makes it easier to build connected systems.--### How can I use gRPC with Tyk?--Using Tyk with your gRPC client and server is very easy. Since gRPC uses HTTP/2, you need to enable it by setting `enable_http2=true` for `Downstream-Tyk` and `proxy_enable_http2=true` for `Tyk-Upstream` connections. You also need to set your `listen_path` in your API definitions. 

Not required for plugins.

joshblakeley

comment created time in a month

startedlawl/NoiseTorch

started time in a month

startedcoreruleset/modsecurity-crs-docker

started time in a month

startedTykTechnologies/custom-plugins

started time in a month

startedSpiderLabs/owasp-modsecurity-crs

started time in a month

push eventmatiasinsaurralde/tyk

Leonid Bugaev

commit sha 483e56febd723eade7d7bfb86963592257e41216

Single API load crash should not affect all APIs (#3197)

view details

Leonid Bugaev

commit sha 6bf55b8aad0af88a1961729332d941251d3fce0c

Fix/api panics (#3198) If panic has happened it should print backtrace

view details

Leonid Bugaev

commit sha 9455d46c6e9531250f8d7da46e1f482e29d1b113

Show API details during panic (#3199) The previous change used wrong Spec object

view details

Matias Insaurralde

commit sha b638cf3aa4f3c4c3953344de69fda962c01832bb

plugin-compiler: use a dynamic package path (#3196) Workaround for #3195. To test this without having to rebuild the `tyk-plugin-compiler` image, the `build.sh` can be replaced on runtime using the volume flag: ``` $ docker run --rm -v `pwd`/build.sh:/build.sh -v `pwd`:/plugin-source tykio/tyk-plugin-compiler:v2.9.4.2 plugin1.so ``` With this fix, the scenario described in the original ticket will perform as expected, allowing the usage of multiple Docker-built Go plugins.

view details

push time in a month

startedtin-cat/emailqueue

started time in a month

push eventmatiasinsaurralde/tyk-pump

Sedky Abou-Shamalah

commit sha e63f0c76eebf1a55c0e7cc61e880d0c3a7f10507

Update README.md

view details

Sedky Abou-Shamalah

commit sha b59d43badea9a0b6cf784f3b3879addf5d34f2df

Update README.md

view details

Sedky Abou-Shamalah

commit sha fb3a1da9ef103fe09151ff9ebd5956cd02fba5e9

Update README.md

view details

Zavierazo

commit sha 402dab85b1fcf0c5f28433292004598425bd20b1

Fix health endpoint to be published outside server. Currently only available from localhost (#267) Co-authored-by: xortiz <xortiz@hotelbeds.com> ## Description Fix healthCheck to be accessible from outside of the server. Currently is only accessible from localhost ## Related Issue fixes #266 ## Motivation and Context Be able to check health of application from outside with monitoring tools. ## How This Has Been Tested Locally test with http://localhost:8080/health http://10.160.255.103:8080/health

view details

Sedky Abou-Shamalah

commit sha 91bcaf6268b0aeef9d10253105db6ed9922fcbe7

Update README.md

view details

Sedky Abou-Shamalah

commit sha 4701a0d832745170191b9df85f807759075b5a66

Merge pull request #268 from TykTechnologies/sedkis-patch-1 Update README.md

view details

Tomas Buchaillot

commit sha aad47bbe3acf74675bf522305be186afa4eb5f07

hybrid pump reconnect (#265) <!-- Provide a general summary of your changes in the Title above --> ## Description <!-- Describe your changes in detail --> The idea of this PR is to add an RPC connection validation on each Hybrid pump write and a reconnection mechanism. ## Related Issue <!-- This project only accepts pull requests related to open issues --> <!-- If suggesting a new feature or change, please discuss it in an issue first --> <!-- If fixing a bug, there should be an issue describing it with steps to reproduce --> <!-- Please link to the issue here --> Closes #264 . ## Motivation and Context <!-- Why is this change required? What problem does it solve? --> Solves the problem that happens when the RPC connection is lost (network issues / idle connection) and the hybrid pump doesn't try to reconnect again. ## How This Has Been Tested <!-- Please describe in detail how you tested your changes --> <!-- Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. --> - Run hybrid pump - Wait for 24hrs+ without writing any record OR disconnect the machine network. - Write an analytic record. It should show you `ERROR hybrid-pump: Failed to login to RPC server, trying to reconnect...` and automatically reconnect, the next record you write it should be written fine. ## Screenshots (if appropriate) ## Types of changes <!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: --> - [X] Bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) ## Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply --> <!-- If you're unsure about any of these, don't hesitate to ask; we're here to help! --> - [x] Make sure you are requesting to **pull a topic/feature/bugfix branch** (right side). If pulling from your own fork, don't request your `master`! - [X] Make sure you are making a pull request against the **`master` branch** (left side). Also, you should start *your branch* off *our latest `master`*. - [ ] My change requires a change to the documentation. - [ ] If you've changed APIs, describe what needs to be updated in the documentation. - [ ] I have updated the documentation accordingly. - [X] Modules and vendor dependencies have been updated; run `go mod tidy && go mod vendor` - [ ] I have added tests to cover my changes. - [X] All new and existing tests passed. - [X] Check your code additions will not fail linting checks: - [X] `go fmt -s` - [X] `go vet`

view details

Zavierazo

commit sha 20aa2600ed893f41840c22dd376e7b5a16c2faaf

Add new filters based on response_code's (#270) Add new filters based on response_code's ## Description Currently we have the necessity to be able to only send with pump request with errors. For that we need to implement this new filter to be able to remove all 200 response code. ## Motivation and Context Decrease space required to store all data of analytics . Request with 200 are not important to be monitorized and also are the 99% of traffic. We have some problems regarding sizing of our cluster of kafka and we want to reduce the load. ## How This Has Been Tested -Unit test -Start pump in local with same config of our staging environment with the new filter.

view details

Alok G Singh

commit sha 9fc5a00a9d77b103c34011c766898bbbfdcaf844

Build images on push to integration-**, qa/*, - release-** and master (#262) Build bare-bones docker image on pushes to - integration-** - qa/* - release-** - master ## Related Issues https://github.com/TykTechnologies/internal/issues/64 ## Motivation and Context Docker images are built after packages are built. This is too late in the process to effectively perform tests. [Big picture](https://docs.google.com/document/d/18fRiUOiT9r_zsmRDSNt1xhuFt7dZb_8wZKukLXJZOnA/) ## How This Has Been Tested See [run](https://github.com/TykTechnologies/tyk-pump/runs/742277491?check_suite_focus=true) ## Types of changes <!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: --> - [ ] Bug fix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) ## Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply --> <!-- If you're unsure about any of these, don't hesitate to ask; we're here to help! --> - [x] Make sure you are requesting to **pull a topic/feature/bugfix branch** (right side). If pulling from your own fork, don't request your `master`! - [x] Make sure you are making a pull request against the **`master` branch** (left side). Also, you should start *your branch* off *our latest `master`*.

view details

push time in a month

PR opened TykTechnologies/tyk

plugin-compiler: use a dynamic package path

Workaround for #3195.

To test this without having to rebuild the tyk-plugin-compiler image, the build.sh can be replaced on runtime using the volume flag:

$ docker run --rm -v `pwd`/build.sh:/build.sh -v `pwd`:/plugin-source tykio/tyk-plugin-compiler:v2.9.4.2 plugin1.so

With this fix, the scenario described in the original ticket will perform as expected, allowing the usage of multiple Docker-built Go plugins.

+2 -1

0 comment

1 changed file

pr created time in a month

issue openedTykTechnologies/tyk

Not able to load multiple native Go plugins

Branch/Environment/Version

  • Branch/Version: All
  • Environment: All

Describe the bug When using the tyk-plugin-compiler to build plugins, all generated shared libraries use the same package path. This causes an error when trying to load more than one plugin, for example, when plugin1 is used in api1 andplugin2is used inapi2`:

level=error msg="Could not load Go-plugin" error="plugin.Open("/tmp/test-plugin/plugin1"): plugin already loaded" mwPath="/tmp/test-plugin/plugin1.so" mwSymbolName=AddFooBarHeader

It's important to note that this error isn't related to duplicate handler/hook names but it's caused by a common package path that's used in all Docker-based builds, even if the host directory is a different one, it's mounted under the same path as a Docker volume on every build.

As a workaround my suggestion is to use a dynamic package path, this is also mentioned in golang discussions.

Reproduction steps Steps to reproduce the behavior:

  1. Create a directory for your sample plugin (/tmp/sample-plugin) and a file named plugin.go with the following contents:
package main

import (
  "net/http"
)

// AddFooBarHeader adds custom "Foo: Bar" header to the request
func AddFooBarHeader(rw http.ResponseWriter, r *http.Request) {
  r.Header.Add("Foo", "Bar")
}

func main() {}
  1. Build a Go plugin:
$ docker run --rm -v `pwd`:/plugin-source tykio/tyk-plugin-compiler:v2.9.4.2 plugin1.so 
  1. Build a second version:
$ docker run --rm -v `pwd`:/plugin-source tykio/tyk-plugin-compiler:v2.9.4.2 plugin2.so 
  1. Prepare two API definitions, one will load plugin.so, the other one will try to load plugin2.so:

API 1:

    "custom_middleware": {
        "pre": [],
        "post_key_auth": [],
        "auth_check": {},
        "post": [
            {
                "name": "AddFooBarHeader",
                "path": "/tmp/test-plugin/plugin1.so"
            }
        ],
        "driver": "goplugin"
    },

API 2:

    "custom_middleware": {
        "pre": [],
        "post_key_auth": [],
        "auth_check": {},
        "post": [
            {
                "name": "AddFooBarHeader",
                "path": "/tmp/test-plugin/plugin2.so"
            }
        ],
        "driver": "goplugin"
    },
  1. The following error will be triggered:
level=error msg="Could not load Go-plugin" error="plugin.Open("/tmp/test-plugin/plugin1"): plugin already loaded" mwPath="/tmp/test-plugin/plugin1.so" mwSymbolName=AddFooBarHeader
  1. After the initialization process, only one of the APIs would be executing the plugin. The plugin will be disabled for one of the APIs because of the above error.
  2. For testing I use the following docker run string, where the apps directory contains the two API definitions described above:
docker run --name tyk_gateway -p 8080:8080 -v $(pwd)/tyk.conf:/opt/tyk-gateway/tyk.conf -v $(pwd)/apps:/opt/tyk-gateway/apps -v /tmp/test-plugin:/tmp/test-plugin tykio/tyk-gateway

Expected behavior It should be possible to load multiple Go plugins when using the tyk-plugin-compiler.

created time in a month

push eventmatiasinsaurralde/tyk

Matias Insaurralde

commit sha a5e9693d4cb7dd2d8ae34b50ad3a826a9261dede

plugin-compiler: use a dynamic package path

view details

push time in a month

create barnchmatiasinsaurralde/tyk

branch : goplugin-ldflags

created branch time in a month

push eventmatiasinsaurralde/tyk

Sergey Petrunin

commit sha 6113c515e7f165ff02089aa42784fd8e416d3664

remove swallowing of originalKey variable (#3184) Fix https://github.com/TykTechnologies/tyk-analytics/issues/1988 <!-- Provide a general summary of your changes in the Title above --> - remove swallowing of originalKey variable - add preserving basic auth data when it was set previously ## Description <!-- Describe your changes in detail --> ## Related Issue <!-- This project only accepts pull requests related to open issues --> <!-- If suggesting a new feature or change, please discuss it in an issue first --> <!-- If fixing a bug, there should be an issue describing it with steps to reproduce --> <!-- Please link to the issue here --> ## Motivation and Context <!-- Why is this change required? What problem does it solve? --> ## How This Has Been Tested <!-- Please describe in detail how you tested your changes --> <!-- Include details of your testing environment, and the tests you ran to see how your change affects other areas of the code, etc. --> ## Screenshots (if appropriate) ## Types of changes <!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: --> - [x] Bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Refactoring or add test (improvements in base code or adds test coverage to functionality) ## Checklist <!-- Go over all the following points, and put an `x` in all the boxes that apply --> <!-- If you're unsure about any of these, don't hesitate to ask; we're here to help! --> - [x] Make sure you are requesting to **pull a topic/feature/bugfix branch** (right side). If pulling from your own fork, don't request your `master`! - [x] Make sure you are making a pull request against the **`master` branch** (left side). Also, you should start *your branch* off *our latest `master`*. - [ ] My change requires a change to the documentation. - [ ] If you've changed APIs, describe what needs to be updated in the documentation. - [ ] If new config option added, ensure that it can be set via ENV variable - [ ] I have updated the documentation accordingly. - [ ] Modules and vendor dependencies have been updated; run `go mod tidy && go mod vendor` - [ ] When updating library version must provide reason/explanation for this update. - [ ] I have added tests to cover my changes. - [ ] All new and existing tests passed. - [ ] Check your code additions will not fail linting checks: - [ ] `go fmt -s` - [ ] `go vet`

view details

Alok G Singh

commit sha aa6c30b126bb1afaedf9f570e221417e249c0acb

Build integration image (#3128) Build bare-bones docker image on pushes to - integration-** - qa/* - release-** - master ## Caveats Absolute paths are not supported on the GH runners. Code is checked out into `/__w/tyk/tyk`. goplugins need to be tested. The image does not have any optional dependencies like Python. Will add dependencies as the image grows support for every type of test. ## Related Issue https://github.com/TykTechnologies/internal/issues/64 [Big picture](https://docs.google.com/document/d/18fRiUOiT9r_zsmRDSNt1xhuFt7dZb_8wZKukLXJZOnA/edit#) ## Motivation and Context Docker images are built after packages are built. This is too late in the process to effectively perform tests. ## How This Has Been Tested See [run](https://github.com/TykTechnologies/tyk/runs/741527603?check_suite_focus=true) Image can be checked out. ## Types of changes <!-- What types of changes does your code introduce? Put an `x` in all the boxes that apply: --> - [ ] Bug fix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) ## Checklist - [x] Make sure you are requesting to **pull a topic/feature/bugfix branch** (right side). If pulling from your own fork, don't request your `master`! - [x] Make sure you are making a pull request against the **`master` branch** (left side). Also, you should start *your branch* off *our latest `master`*. - [ ] My change requires a change to the documentation. - [ ] If you've changed APIs, describe what needs to be updated in the documentation. - [ ] I have updated the documentation accordingly. - [ ] Modules and vendor dependencies have been updated; run `go mod tidy && go mod vendor` - [ ] When updating library version must provide reason/explanation for this update. - [ ] I have added tests to cover my changes. - [ ] All new and existing tests passed. - [ ] Check your code additions will not fail linting checks: - [ ] `go fmt -s` - [ ] `go vet`

view details

push time in a month

more