profile
viewpoint
Matias Insaurralde matiasinsaurralde @TykTechnologies Paraguay https://matias.insaurral.de/ software developer (most of the time), infosec enthusiast, human being, self-taught. 24 years.

issue commentTykTechnologies/tyk

gRPC uses full URI and not Path

Quick note on the bug description: looks like it's actually the opposite, at least, when looking at the code, we find r.URL.Path in 2.8.3 and r.URL.String() in 2.8.4. I guess r.URL.String() is fine (in this scenario the user can parse the full URL and extract specific parameters)!

alephnull

comment created time in 19 hours

startedgluc/ahp

started time in 21 hours

startedpyAHP/pyAHP

started time in 21 hours

Pull request review commentTykTechnologies/tyk

log stripping version message in debug mode

 func (s *SuccessHandler) ServeHTTPWithCache(w http.ResponseWriter, r *http.Reque 	if !s.Spec.VersionData.NotVersioned && versionDef.Location == "url" && versionDef.StripPath { 		part := s.Spec.getVersionFromRequest(r) -		log.Info("Stripping version from url: ", part)+		log.Debug("Stripping version from url: ", part)

Perhaps use URL (uppercase) as we do in other similar log messages?

gernest

comment created time in a day

pull request commentTykTechnologies/tyk

Supporting dlpython

Note that coprocess tag isn't required anymore. See here.

For Lua: go build -tags 'lua'.

alephnull

comment created time in a day

startedmicrosoft/AirSim

started time in 2 days

issue openedTykTechnologies/tyk

Key is visible in logs when session is recreated

Branch/Environment/Version

  • Branch/Version: All
  • Environment: All

Describe the bug I don't have much context on the scenario of this bug, but seems that middleware.go doesn't obfuscate the key when logging "Recreating session for key".

Expected behavior My guess is that we should modify this to use enable_key_logging and obfuscateKey function.

created time in 2 days

issue openedTykTechnologies/tyk

JSVM authentication broken when using bundles

Branch/Environment/Version

  • Branch/Version: Master
  • Environment: On-prem

Describe the bug I've been testing JSVM authentication with a bundle, seems that DynamicMiddleware doesn't get initialized with the right flag, when using an authentication hook: https://github.com/TykTechnologies/tyk/blob/master/gateway/middleware.go#L63

In api_loader.go we have the following code:

		if ottoAuth {
			logger.Info("----> Checking security policy: JS Plugin")

			authArray = append(authArray, createDynamicMiddleware(mwAuthCheckFunc.Name, true, false, baseMid))
		}

This call will always initialize DynamicMiddleware with Auth set to false. To fix this we could extend createDynamicMiddleware and add an argument. After a quick test, I've found that this fixes the issue. bundle.zip

Reproduction steps Steps to reproduce the behavior:

  1. Configure an API with the following settings:
    "custom_middleware_bundle": "bundle.zip",
    "enable_coprocess_auth": true,
  1. Serve the attached bundle (bundle.zip).

Actual behavior The middleware gets called but the user receives the following error:

% curl localhost:8080/quickstart/ -H 'Authorization: foobar'
{
    "error": "Session state is missing or unset! Please make sure that auth headers are properly applied"
}

Expected behavior When sending foobar, the authentication should be successful.

created time in 3 days

push eventmatiasinsaurralde/tyk

Furkan Senharputlu

commit sha 25efa49165cf8ec0083082b4485a44e127dce212

Fix TestKeyHandler_HashingDisabled test (#2507)

view details

Furkan Senharputlu

commit sha 3898719cb3e671698504968f9d3a2610354516d6

Fix GetKeys filter in RedisCluster (#2505) Fixes https://github.com/TykTechnologies/tyk/issues/2508

view details

Furkan Senharputlu

commit sha 0780a7f01cec9bedf52c5cec248f685f9036379f

Handle B64 org ID match for key listing (#2509) Expands #2505

view details

Matias Insaurralde

commit sha d80fd87800798c2746dc5f48621c5296cff14b51

Fix CP tests (#2506) Fixes tests for Go 1.10

view details

push time in 3 days

startedtchx84/tg-rs-upvote-bot

started time in 3 days

PR opened TykTechnologies/tyk

Fix CP tests (WIP)
+2 -0

0 comment

2 changed files

pr created time in 7 days

create barnchmatiasinsaurralde/tyk

branch : fix-go110-tests

created branch time in 7 days

push eventmatiasinsaurralde/tyk

Leonid Bugaev

commit sha 3b79ef26157a9701d38dad9819c55a69dae9e791

Feature/multi policy addon (#2503) Updates "Update policy" API used by the dashboard developers screen Additionally, it adds "key preview" screen, which you can use to similar how policies combine together before creating a key.

view details

push time in 7 days

PR opened TykTechnologies/tyk

Release 2.9 / dlpython

Cherry picked from 5132c8295c0ca9dba56825a03b7a2fe458bfab3e, for 2.9.

+1091 -3854

0 comment

41 changed files

pr created time in 8 days

create barnchmatiasinsaurralde/tyk

branch : release-2.9-dlpython

created branch time in 8 days

create barnchmatiasinsaurralde/tyk

branch : release-dlpython

created branch time in 8 days

push eventmatiasinsaurralde/tyk

Furkan Senharputlu

commit sha f7b6cb84e516a48559322b81da5aff8ea2b16c2e

Govendor fetch drl to fix race condition (#2487)

view details

Alok G Singh

commit sha 040507e005a2097546332ee60f6f7acfdf17e7dc

Updating supported versions (#2494) precise was EOL a couple of years ago bionic is the newest LTS jessie is reaching EOL but not yet

view details

Matias Insaurralde

commit sha 5132c8295c0ca9dba56825a03b7a2fe458bfab3e

Remove build tags and use dynamic Python loader (#1875) This solves #1283 (as discussed build tags in this scenario are no longer needed). - Most of CPython glue code was moved and simplified into its own package (`dlpython` directory). - The dynamic Python loader uses `python-config` to find available Python versions in the system, the latest version is selected by default. - If the user wants to override a specific version, it's possible to set `python_version` under `coprocess_options`, specifying `3.7` for example. - By default, when using `go build`, the resulting binary will support Python and gRPC. Python will be loaded only when it's available in the system and a Python plugin is used. - The Lua build has its own build tag: `go build -tags 'lua'`. - It's possible to have multiple APIs using different types of plugins at the same time, e.g. API 1 with a gRPC plugin and API 2 with Python.

view details

push time in 9 days

push eventmatiasinsaurralde/tyk

Matias Insaurralde

commit sha cf2c8f090ac931808e2b7d7acf5af9cd0ce86dd1

coprocess: load Python only when it's in use

view details

push time in 11 days

Pull request review commentTykTechnologies/tyk

[Feature] Add policies rest api

 func apiHandler(w http.ResponseWriter, r *http.Request) { 	doJSONWrite(w, code, obj) } +func policiesHandler(w http.ResponseWriter, r *http.Request) {+	policyID := mux.Vars(r)["policyID"]++	var obj interface{}+	var code int++	switch r.Method {+	case "GET":+		if policyID != "" {+			log.Debug("Requesting Policy definition for", policyID)+			obj, code = handleGetPolicy(policyID)+		} else {+			log.Debug("Requesting Policies list")+			obj, code = handleGetPoliciesList()+		}+	case "POST":

http.MethodPost here

williamokano

comment created time in 11 days

Pull request review commentTykTechnologies/tyk

[Feature] Add policies rest api

 func apiHandler(w http.ResponseWriter, r *http.Request) { 	doJSONWrite(w, code, obj) } +func policiesHandler(w http.ResponseWriter, r *http.Request) {+	policyID := mux.Vars(r)["policyID"]++	var obj interface{}+	var code int++	switch r.Method {+	case "GET":

Better to use http.MethodGet and similar constants.

williamokano

comment created time in 11 days

Pull request review commentTykTechnologies/tyk

[Feature] Add policies rest api

 func apiHandler(w http.ResponseWriter, r *http.Request) { 	doJSONWrite(w, code, obj) } +func policiesHandler(w http.ResponseWriter, r *http.Request) {+	policyID := mux.Vars(r)["policyID"]++	var obj interface{}+	var code int++	switch r.Method {+	case "GET":+		if policyID != "" {+			log.Debug("Requesting Policy definition for", policyID)+			obj, code = handleGetPolicy(policyID)+		} else {+			log.Debug("Requesting Policies list")+			obj, code = handleGetPoliciesList()+		}+	case "POST":+		log.Debug("Creating new policy")+		obj, code = handleAddOrUpdatePolicy(policyID, r)+	case "PUT":

http.MethodPut here

williamokano

comment created time in 11 days

push eventmatiasinsaurralde/tyk

Matias Insaurralde

commit sha fec52f337218641edf6db5b467089fee9b8d4ad7

CI: switch Python version

view details

push time in 16 days

push eventmatiasinsaurralde/tyk

Matias Insaurralde

commit sha 0d7be292ee4eedec68397f649890121bbfc62a96

CI: fix tests

view details

push time in 16 days

push eventmatiasinsaurralde/tyk

Matias Insaurralde

commit sha 519ab4d39ef01db68997e9a03773e15b9f132b63

Update Travis configuration

view details

push time in 16 days

push eventmatiasinsaurralde/tyk

Matias Insaurralde

commit sha eb4188997973b7d35d31d9b0589e07e5db09e6c8

coprocess: remove default Python version from tests

view details

push time in 16 days

push eventmatiasinsaurralde/tyk

Matias Insaurralde

commit sha 5fb8663ede764cf2e0c33b8f2c954e49ccef4ba1

CI: fix tests

view details

push time in 16 days

push eventmatiasinsaurralde/tyk

Matias Insaurralde

commit sha 8804a05176e3a62a6f04c21c270b98cf2fd225c0

CI: fix tests

view details

push time in 16 days

push eventmatiasinsaurralde/tyk

Geofrey Ernest

commit sha a290e4046554011673c1ef3667004e29e3122cdf

account for data expiration on orgid failed cache lookup (#2463) Fixes #2370

view details

Leonid Bugaev

commit sha 578a81206bcb121f05aa9bc19c6840b9645c0755

Gofmt

view details

Geofrey Ernest

commit sha 79b6392b75f6394db6988931324e2ffcba69e97c

Fix Proxy.ListenPath stripping (#2466) Fixes #2421

view details

Furkan Senharputlu

commit sha b9b98bf899cfcdc4cc83a4a53695d5546043380a

Save version in redis (#2450) Related to https://github.com/TykTechnologies/product/issues/42

view details

Komal Sukhani

commit sha b818a2b88a256cce6e48d07a97dab86f670f37b1

Return error if Service Discovery endpoint returns invalid json response (#2451) If Service Discovery endpoint returns invalid JSON response, gateway panics. Fixed the issue by returning from the method if an error occurs

view details

Furkan Senharputlu

commit sha 3bdce7650221f35df4160514ade7bf34f1012e17

Test refactoring (#2469) * Make TestHttpAny const public * Make test server pointer

view details

Ilija Bojanovic

commit sha 6e8bd8331de191a36f59023e67cacc6b1031f5ea

Typo fix in template (#2472)

view details

Furkan Senharputlu

commit sha fbf93dfd4a28021a998d16c8d41f8dc6b3279568

Add JSVM metadata update test (#2474) Fixes #2471

view details

Leonid Bugaev

commit sha b8b0e18d8b0ac825dec3ff541415c65ce955f1b0

Add support for combining policies with multiple rate/quotas (#2462) If it finds that Key has policies with intersection ACL partitions rates/quotas will have separate counters per policy. So now you can actually safely mix policies in any combination. If it finds that Key has policies with the same ACL, it will merge them together. So now it is possible to have one "read" policy (with permissions per path/method), second "write" policy, and you can mix them together, just by assigning multiple policies to the key. Fix https://github.com/TykTechnologies/tyk-analytics/issues/1369

view details

dencoded

commit sha 4c1eb530ee3a394a717e7707f89842bd1b7e1053

support of imported keys with new format added (#2473) added changes for https://github.com/TykTechnologies/product/issues/165 I required to modify tests a lot as now operation with key requires carefully set and matched orgID

view details

Geofrey Ernest

commit sha 56de48ca479aa702314127ee0e33338a3607d3f8

tcp proxy (#2426) The current code adds initial code for TCP proxy which multiplex to different services. Additionally refactored all code related to how we start a web server, in particular now you can run multiple APIs on different ports, and Tyk will dynamically open or close port listeners. Added 2 new fields to API definition: `listen_port` and `protocol`. Valid protocol values: "", "http", "https", "tcp", "tls". By default, protocol is selected based on `http_server_options.use_ssl`, and can be either http or https. Additionally in order to tell that your upstream should be `tcp` or `tls` one, in target URL you can specify protocol like this: "tls://upstream:<port>". So you can have GW listening on TLS, but pointing to TCP upstreaming, or the other way. Example service description: ``` "listen_port": 30001, "protocol": "tls", "certificate": ["<cert-id>"], "proxy": { "target_url": "tls://upstream:9191" } ``` All the TLS related features, like mutual TLS or certificate pinning work as expected. Adding "listen_port", means that you can now start HTTPS server on one port, HTTP on another port, and some TCP services on another port as well. The only requirement that each port should serve the same protocol (GW has checks preventing it). Additionally, TCP proxying, support multiplexing based on SNI information, e.g. you can serve multiple TCP services on different domains, pointing to different upstream. - [x] Analytics support - [x] ~Way to specific fixed ports and protocols they are support~Way to specific fixed ports and protocols they don't support - [x] Support load balancing and service discovery - [x] Health checks - [x] Graceful restarts - [x] Proxy protocol https://github.com/TykTechnologies/tyk/issues/2300

view details

Ahmet Soormally

commit sha 2b9bdf42075b21e5eecf062b62f8056b17580147

Makefile to make dev env a bit easier (#2479) Development Environment helper Makefile Could also be used by CI?

view details

Furkan Senharputlu

commit sha 2a5ddcc0dd2249760a55ce52c57d36506e905166

Make NodeID setting and getting public (#2482)

view details

Furkan Senharputlu

commit sha 2a37a6e44ed63befd55e0d70b22723086d760fcb

Fix broken tests (#2483)

view details

Furkan Senharputlu

commit sha 296c71114101fe3f1073b8bc1dd551a0b0c3884b

Make doReload func public (#2484)

view details

Alok G Singh

commit sha 3173c58cba7649b2911ad4173908d0207380ee9d

Adding debian/stretch (#2486)

view details

Matias Insaurralde

commit sha 03889917d93bf0df2402c68f178fe2c63cbce92f

coprocess: use dlpython

view details

Matias Insaurralde

commit sha 2854258d54ffe73d379e71fed3e3644ea4c23131

coprocess: reimplement gateway bindings using ctypes

view details

Matias Insaurralde

commit sha 62dfce1b3a46b41bf6a2774b9c32055eaae35de8

coprocess: fix logging

view details

Matias Insaurralde

commit sha 85a2f23487347316735ec05bcc93f15d05f645f1

coprocess: minor fixes

view details

push time in 16 days

push eventmatiasinsaurralde/tyk

Geofrey Ernest

commit sha a290e4046554011673c1ef3667004e29e3122cdf

account for data expiration on orgid failed cache lookup (#2463) Fixes #2370

view details

Leonid Bugaev

commit sha 578a81206bcb121f05aa9bc19c6840b9645c0755

Gofmt

view details

Geofrey Ernest

commit sha 79b6392b75f6394db6988931324e2ffcba69e97c

Fix Proxy.ListenPath stripping (#2466) Fixes #2421

view details

Furkan Senharputlu

commit sha b9b98bf899cfcdc4cc83a4a53695d5546043380a

Save version in redis (#2450) Related to https://github.com/TykTechnologies/product/issues/42

view details

Komal Sukhani

commit sha b818a2b88a256cce6e48d07a97dab86f670f37b1

Return error if Service Discovery endpoint returns invalid json response (#2451) If Service Discovery endpoint returns invalid JSON response, gateway panics. Fixed the issue by returning from the method if an error occurs

view details

Furkan Senharputlu

commit sha 3bdce7650221f35df4160514ade7bf34f1012e17

Test refactoring (#2469) * Make TestHttpAny const public * Make test server pointer

view details

Ilija Bojanovic

commit sha 6e8bd8331de191a36f59023e67cacc6b1031f5ea

Typo fix in template (#2472)

view details

Furkan Senharputlu

commit sha fbf93dfd4a28021a998d16c8d41f8dc6b3279568

Add JSVM metadata update test (#2474) Fixes #2471

view details

Leonid Bugaev

commit sha b8b0e18d8b0ac825dec3ff541415c65ce955f1b0

Add support for combining policies with multiple rate/quotas (#2462) If it finds that Key has policies with intersection ACL partitions rates/quotas will have separate counters per policy. So now you can actually safely mix policies in any combination. If it finds that Key has policies with the same ACL, it will merge them together. So now it is possible to have one "read" policy (with permissions per path/method), second "write" policy, and you can mix them together, just by assigning multiple policies to the key. Fix https://github.com/TykTechnologies/tyk-analytics/issues/1369

view details

dencoded

commit sha 4c1eb530ee3a394a717e7707f89842bd1b7e1053

support of imported keys with new format added (#2473) added changes for https://github.com/TykTechnologies/product/issues/165 I required to modify tests a lot as now operation with key requires carefully set and matched orgID

view details

Geofrey Ernest

commit sha 56de48ca479aa702314127ee0e33338a3607d3f8

tcp proxy (#2426) The current code adds initial code for TCP proxy which multiplex to different services. Additionally refactored all code related to how we start a web server, in particular now you can run multiple APIs on different ports, and Tyk will dynamically open or close port listeners. Added 2 new fields to API definition: `listen_port` and `protocol`. Valid protocol values: "", "http", "https", "tcp", "tls". By default, protocol is selected based on `http_server_options.use_ssl`, and can be either http or https. Additionally in order to tell that your upstream should be `tcp` or `tls` one, in target URL you can specify protocol like this: "tls://upstream:<port>". So you can have GW listening on TLS, but pointing to TCP upstreaming, or the other way. Example service description: ``` "listen_port": 30001, "protocol": "tls", "certificate": ["<cert-id>"], "proxy": { "target_url": "tls://upstream:9191" } ``` All the TLS related features, like mutual TLS or certificate pinning work as expected. Adding "listen_port", means that you can now start HTTPS server on one port, HTTP on another port, and some TCP services on another port as well. The only requirement that each port should serve the same protocol (GW has checks preventing it). Additionally, TCP proxying, support multiplexing based on SNI information, e.g. you can serve multiple TCP services on different domains, pointing to different upstream. - [x] Analytics support - [x] ~Way to specific fixed ports and protocols they are support~Way to specific fixed ports and protocols they don't support - [x] Support load balancing and service discovery - [x] Health checks - [x] Graceful restarts - [x] Proxy protocol https://github.com/TykTechnologies/tyk/issues/2300

view details

Ahmet Soormally

commit sha 2b9bdf42075b21e5eecf062b62f8056b17580147

Makefile to make dev env a bit easier (#2479) Development Environment helper Makefile Could also be used by CI?

view details

Furkan Senharputlu

commit sha 2a5ddcc0dd2249760a55ce52c57d36506e905166

Make NodeID setting and getting public (#2482)

view details

Furkan Senharputlu

commit sha 2a37a6e44ed63befd55e0d70b22723086d760fcb

Fix broken tests (#2483)

view details

Furkan Senharputlu

commit sha 296c71114101fe3f1073b8bc1dd551a0b0c3884b

Make doReload func public (#2484)

view details

Alok G Singh

commit sha 3173c58cba7649b2911ad4173908d0207380ee9d

Adding debian/stretch (#2486)

view details

push time in 16 days

push eventmatiasinsaurralde/tyk

Matias Insaurralde

commit sha 9c1ad090bf4e7c6092e17e2d01f5aba5ab6a7e00

coprocess: enhance exception handler

view details

push time in 16 days

push eventmatiasinsaurralde/tyk

Matias Insaurralde

commit sha 8d6c59879783e137488325e89183bebe8373ac44

coprocess: check PythonPathPrefix before loading Python

view details

Matias Insaurralde

commit sha 4d330ae413b48cfb20fd6456c29856d3267ed9e5

coprocess: extend python.FindPythonConfig call

view details

Matias Insaurralde

commit sha f8549038779a7ef2ca4b67be538af9df15c1f78a

coprocess: enhance FindPythonConfig functionality, use a more appropriate regular expression

view details

push time in 16 days

push eventmatiasinsaurralde/tyk

Matias Insaurralde

commit sha ed31987005c085af00bf37ac4a7256207484601e

ldd wrapper etc

view details

push time in 21 days

startedyookoala/realpath

started time in 21 days

startedmafintosh/ldd

started time in 21 days

push eventmatiasinsaurralde/tyk

Matias Insaurralde

commit sha 486c5bf70166c31615b22f7012543b54f337c8ce

test

view details

push time in 21 days

push eventmatiasinsaurralde/tyk

Matias Insaurralde

commit sha d4283ce86c56b41146626f53896892c832ec0eba

temp

view details

push time in 21 days

push eventmatiasinsaurralde/tyk

Matias Insaurralde

commit sha 7d3bbe8ed430fd2c7442f17abb8b7112ca19cb1e

coprocess: extend FindPythonConfig to cover more cases

view details

push time in 21 days

push eventmatiasinsaurralde/tyk

Matias Insaurralde

commit sha 8e1a053c04cf706f2b8f45b39e0a3308ce834c29

coprocess: refactor code to keep Lua build under a separate tag

view details

push time in 23 days

startedpfalcon/picoweb

started time in 23 days

pull request commentTykTechnologies/tyk

Remove build tags and use dynamic Python loader

  • The dispatcher logic was refactored in order to allow multiple dispatchers in the same process.

Does this mean that a gRPC plugin and a python plugin can co-exist in the same gateway? e.g. one API definition can target a gRPC plugin, and another can target Python?

Yes!

matiasinsaurralde

comment created time in a month

pull request commentTykTechnologies/tyk

Remove build tags and use dynamic Python loader

Updates:

  • Tests are passing: go test -timeout 30s github.com/TykTechnologies/tyk/coprocess/python and go test -timeout 30s github.com/TykTechnologies/tyk/coprocess/grpc.
  • The dispatcher logic was refactored in order to allow multiple dispatchers in the same process.
  • gRPC and Python can work as part of the same binary, no build tags are required, just go build. Also removed build tags from these tests.
  • Updated schema for the linter to accept python_version.

Missing items to fix today:

  • Fix/remove 'coprocess' tests.
  • Fix Lua build and make it work under its own build tag.
matiasinsaurralde

comment created time in a month

push eventmatiasinsaurralde/tyk

Matias Insaurralde

commit sha f8b9dbf257673c8d58fee540674204edcc9d99d8

bin: update CI script

view details

push time in a month

push eventmatiasinsaurralde/tyk

Matias Insaurralde

commit sha b86901a2a3375208fd0f41d93b0afab98d135f3c

coprocess: restore loader logging

view details

push time in a month

push eventmatiasinsaurralde/tyk

Matias Insaurralde

commit sha 230f26fa2f5ef0c579e2d8084426cc57793a1e1a

coprocess: only use LockOSThread, use object length to reconstruct the PB object

view details

Matias Insaurralde

commit sha 2acbc4f6d88faa244d16b4390f5fb349d4e3823e

coprocess: remove gRPC build tags from tests

view details

Matias Insaurralde

commit sha b39b26c7e9f31eb4da7303fe796b5bf8fe4b65ae

dlpython: extend PyBytesAsString helper

view details

Matias Insaurralde

commit sha 31352383a2f320dabc6d60afb988350afa7af52c

coprocess: remove Python build tag from bundle tests

view details

Matias Insaurralde

commit sha e1da2005ca3e6ba0fb799cf915a5d4825f38060c

cli: update schema to include "python_version"

view details

Matias Insaurralde

commit sha 310610d213d14cbadad3a4763ba9b3818d61ce05

coprocess: add default Python version, remove old PB Python samples

view details

push time in a month

push eventmatiasinsaurralde/tyk

Matias Insaurralde

commit sha 5d186bbfe8e30f40a52250935f94ba3ec1f2ffb8

coprocess: merge gRPC and Python build

view details

Matias Insaurralde

commit sha 395e152e450d1eb2b32f2f4716d082c9d0589040

coprocess: additional tweaks, remove GlobalDispatcher references

view details

push time in a month

startedbsm/redeo

started time in a month

pull request commentTykTechnologies/tyk

[WIP] Remove build tags and use dynamic Python loader

I have refactored this in order to include dlpython in the gateway repo. Also:

  • dlpython logging was integrated with Tyk logger.
  • Tests were adjusted on dlpython.testVersion in dlpython/main_test.go needs to be set to an appropriate version in order to run dlpython tests.
  • I have tried to come up with a default Python version load approach but seems that this can be problematic, one of the reasons is that python-config somestimes refer to older Python versions, the second reason is that if we load python-config we don't really know which version we're loading (we only guess this from the filename).
  • We should suggest users to pick a specific version of Python or ship with a default one (every distro might have a different one), for example 3.7.
matiasinsaurralde

comment created time in a month

push eventmatiasinsaurralde/tyk

Matias Insaurralde

commit sha a2717d2980e0fe26a6489a6f557e5b4a63e7e176

coprocess: use internal package

view details

Matias Insaurralde

commit sha ac272ea12b6adb44425c542feb76e8c9a390e990

coprocess: include dlpython package

view details

push time in a month

push eventmatiasinsaurralde/tyk

Geofrey Ernest

commit sha 18c8fc08e583431cfef5fc8acbff4da1df5f2ffe

fix panic for bad integer casting (#2410) HardTimeout expects float64 result but HardTimeoutMeta.Timeout is int This causes the panic when calling spec.CheckSpecMatchesStatus with HardTimeout

view details

Matias Insaurralde

commit sha d8457f9949f576e2b08863e7b6dd83c754f39c23

Support response middleware when using rich plugins (#2321) Added new `response` hook type ``` { "file_list": [ ], "custom_middleware": { "pre": [ { "name": "MyPreHook" } ], "response": [{ "name": "MyResponseHook" }] } } ``` ``` from tyk.decorators import * from gateway import TykGateway as tyk @Hook def MyResponseHook(request, response, session, metadata, spec): response.raw_body = b'newbody' return response ```

view details

Matias Insaurralde

commit sha 6c1264277e5c80a0615e46e5eee9ef981838e3d7

coprocess: use dlpython

view details

Matias Insaurralde

commit sha e30e2343401cb5ba00988fad327f2d979064a503

coprocess: reimplement gateway bindings using ctypes

view details

Matias Insaurralde

commit sha d569f10d80d88337457fa972e5dddcd4db688e58

coprocess: fix logging

view details

Matias Insaurralde

commit sha 747eb4e9b900cf90d26264af4b16f9694f371160

coprocess: minor fixes

view details

push time in a month

push eventmatiasinsaurralde/tyk

Geofrey Ernest

commit sha 18c8fc08e583431cfef5fc8acbff4da1df5f2ffe

fix panic for bad integer casting (#2410) HardTimeout expects float64 result but HardTimeoutMeta.Timeout is int This causes the panic when calling spec.CheckSpecMatchesStatus with HardTimeout

view details

Matias Insaurralde

commit sha d8457f9949f576e2b08863e7b6dd83c754f39c23

Support response middleware when using rich plugins (#2321) Added new `response` hook type ``` { "file_list": [ ], "custom_middleware": { "pre": [ { "name": "MyPreHook" } ], "response": [{ "name": "MyResponseHook" }] } } ``` ``` from tyk.decorators import * from gateway import TykGateway as tyk @Hook def MyResponseHook(request, response, session, metadata, spec): response.raw_body = b'newbody' return response ```

view details

push time in a month

startedtikazyq/crawlab

started time in a month

push eventmatiasinsaurralde/tyk

Leonid Bugaev

commit sha 04972c3735a35a35a69db76caf8a0c6bad0a5616

Fix quota calculation for Key API (#1874) Fixes: https://github.com/TykTechnologies/tyk/issues/1879 It was not using "quota-" prefix.

view details

Josh Blakeley

commit sha aa96862fdbc9668563d4440a7018aeab45b91ba8

linter guaranteed to exit (#1884) Addresses one of issues in #1880 I couldnt consistently reproduce the behaviour but sometime when tyk lint is run tyk process will start up afterwards. This is not the advertised behaviour of the linter currently - so this change guarantees exit. We can change behaviour by attaching flags to the command at a future time.

view details

Matias Insaurralde

commit sha ac789fbc713278cb55e53b4075afe86b0e274534

Make JSVM the default driver (#1888) * api_loader: use mwDriver as the middleware driver, fixes #1887 * coprocess_bundle: only call HandleMiddlewareCache when using rich plugins

view details

dencoded

commit sha 31f70e4847815600520dcefb871dddf54aaea904

per API partitions option added (#1883) Added changes for https://github.com/TykTechnologies/tyk/issues/1783 While looking at this feature I tried to follow ideas: - legacy should continue working without any tricks - we should be able to use the same concept of partitioned policies but with one more edge case when we can specify quota and rate limit in more than one partitioned policy if they have different APIs in ACL - we should be able specify quota and rate limit on API level (ACL item) explicitly when we create policy The idea is to introduce new option "Per API" in partitions section when we create policy. If this option is unchecked - everything should work as before. Also, there is one condition - that new option "Per API" comes to play only when another partition option "Enforce access control" (aka ACL) is checked as this is what tells our logic to merge access rights from different policies into one array in key session when several (partitioned) policies attached to key. Imho it is weird, maybe can be fixed on UI level. Also, I've introduced quota and rate limit on API level (field `limit`) in `AccessDefinition` because: - we need to be able to specify quota and rate limit per ACL item when we create policy - we need to store quota and rate limit per ACL policy even if it was not specified per ACL item but just on policy level - so all ACL items kind of inherit quota and rate from policy and overwrite it if it was specified explicitly while policy was created - when quota and rate on ACL item level inherited from policy `AccessDefinition.Limit` gets new field `SetByPolicy` to true - for informative/debug purposes so we know from where that limit on API level has come from If we have limits on API level we add API ID to storage keys (leaky bucket and other rate limits keys or keys to store quota for key or org session). If we don't have limit specified on API level quota and rate limit should work as they were working before that change. if we try to assign several policies with the same API in ACL it should error. Please note - I didn't include unit tests intentionally (however I've got some) because: - I rewrote this logic several times while exploring different options to implement this feature so unit test were through away thing every time - I am kind of got stuck with the way we test rate limits and quotas (as I receive false positives time to time)

view details

Leonid Bugaev

commit sha 9caced01a3c3e53ef7c5b3780d8a6c2b9a9f1b02

Added new Debugging API (#1896) Added new Tracing API, which allows you to debug API definition, by sending sample request to it. As input, Tracing API require sample request object, and direct API definition. This means that tracing API can be used even without creating API. Or for example, you can test API changes without saving them. Example request: ``` POST /tyk/trace { "request": { "method": "GET", "path": "/get", "headers": { "Header": ["Value"] } }, "spec": { ...valid API definition... } } ``` As output, it gives you response object, and gateway logs as JSON multi-line string. Example output: ``` { "message": "ok" "response": "<raw-http-response-dump>", "logs": "{...}\n{...}" } ``` Logging itself now way more structured, and prepared for easy parsing. Based on the logs you can see: * API loading process: which middleware was loaded or not, issues during parsing and etc. * Middleware logs: order of running, elapsed time, individual MW logs Example of log output: ``` {"level":"debug","msg":"Default JSVM timeout used: 5s","prefix":"jsvm","time":"2018-09-05T17:49:19+03:00"} {"api_id":"test","api_name":"","level":"info","msg":"Loading API","org_id":"","time":"2018-09-05T17:49:19+03:00"} {"api_id":"test","api_name":"","level":"debug","msg":"Initializing API","org_id":"","time":"2018-09-05T17:49:19+03:00"} {"api_id":"test","api_name":"","level":"debug","msg":"Init","mw":"RateCheckMW","org_id":"","time":"2018-09-05T17:49:19+03:00"} {"api_id":"test","api_name":"","level":"debug","msg":"Init","mw":"VersionCheck","org_id":"","time":"2018-09-05T17:49:19+03:00"} {"api_id":"test","api_name":"","level":"info","msg":"Checking security policy: Token","org_id":"","time":"2018-09-05T17:49:19+03:00"} {"api_id":"test","api_name":"","level":"debug","msg":"Init","mw":"AuthKey","org_id":"","time":"2018-09-05T17:49:19+03:00"} {"api_id":"test","api_name":"","level":"debug","msg":"Init","mw":"KeyExpired","org_id":"","time":"2018-09-05T17:49:19+03:00"} {"api_id":"test","api_name":"","level":"debug","msg":"Init","mw":"AccessRightsCheck","org_id":"","time":"2018-09-05T17:49:19+03:00"} {"api_id":"test","api_name":"","level":"debug","msg":"Init","mw":"RateLimitAndQuotaCheck","org_id":"","time":"2018-09-05T17:49:19+03:00"} {"api_id":"test","api_name":"","level":"debug","msg":"Init","mw":"GranularAccessMiddleware","org_id":"","time":"2018-09-05T17:49:19+03:00"} {"api_id":"test","api_name":"","level":"debug","msg":"Init","mw":"VersionCheck","org_id":"","time":"2018-09-05T17:49:19+03:00"} {"api_id":"test","api_name":"","level":"debug","msg":"Init","mw":"KeyExpired","org_id":"","time":"2018-09-05T17:49:19+03:00"} {"api_id":"test","api_name":"","level":"debug","msg":"Init","mw":"AccessRightsCheck","org_id":"","time":"2018-09-05T17:49:19+03:00"} {"api_id":"test","api_name":"","level":"debug","msg":"Rate limit endpoint is: /sampletyk/rate-limits/","org_id":"","time":"2018-09-05T17:49:19+03:00"} {"api_id":"test","api_name":"","level":"debug","msg":"Setting Listen Path: /sample","org_id":"","time":"2018-09-05T17:49:19+03:00"} {"api_id":"test","api_name":"","level":"info","msg":"API Loaded","org_id":"","prefix":"gateway","server_name":"--","time":"2018-09-05T17:49:19+03:00","user_id":"--","user_ip":"--"} {"api_id":"test","api_name":"","level":"debug","msg":"Started","mw":"RateCheckMW","org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00","ts":1536158959332155740} {"api_id":"test","api_name":"","code":200,"level":"debug","msg":"Finished","mw":"RateCheckMW","ns":27925,"org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00"} {"api_id":"test","api_name":"","level":"debug","msg":"Started","mw":"VersionCheck","org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00","ts":1536158959332214289} {"api_id":"test","api_name":"","code":200,"level":"debug","msg":"Finished","mw":"VersionCheck","ns":23042,"org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00"} {"api_id":"test","api_name":"","level":"debug","msg":"Started","mw":"AuthKey","org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00","ts":1536158959332264989} {"api_id":"test","api_name":"","level":"debug","msg":"Querying local cache","mw":"AuthKey","org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00"} {"api_id":"test","api_name":"","level":"debug","msg":"Querying keystore","mw":"AuthKey","org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00"} {"api_id":"test","api_name":"","level":"debug","msg":"Got key","mw":"AuthKey","org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00"} {"api_id":"test","api_name":"","code":200,"level":"debug","msg":"Finished","mw":"AuthKey","ns":1149266,"org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00"} {"api_id":"test","api_name":"","key":"****ecf1","level":"debug","msg":"Started","mw":"KeyExpired","org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00","ts":1536158959333468462} {"api_id":"test","api_name":"","code":200,"key":"****ecf1","level":"debug","msg":"Finished","mw":"KeyExpired","ns":23915,"org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00"} {"api_id":"test","api_name":"","key":"****ecf1","level":"debug","msg":"Started","mw":"AccessRightsCheck","org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00","ts":1536158959333521672} {"api_id":"test","api_name":"","code":200,"key":"****ecf1","level":"debug","msg":"Finished","mw":"AccessRightsCheck","ns":18849,"org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00"} {"api_id":"test","api_name":"","key":"****ecf1","level":"debug","msg":"Started","mw":"RateLimitAndQuotaCheck","org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00","ts":1536158959333632612} {"api_id":"test","api_name":"","code":200,"key":"****ecf1","level":"debug","msg":"Finished","mw":"RateLimitAndQuotaCheck","ns":42329,"org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00"} {"api_id":"test","api_name":"","key":"****ecf1","level":"debug","msg":"Started","mw":"GranularAccessMiddleware","org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00","ts":1536158959333706674} {"api_id":"test","api_name":"","code":200,"key":"****ecf1","level":"debug","msg":"Finished","mw":"GranularAccessMiddleware","ns":17514,"org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00"} ``` Such detailed logs, accessed progmatically, allow some exciting cases like debugging VirtualEndpoint code, and seeing all JS parsing issues, or JS logging data, directly from Dashboard UI. This API also allow you to test authentification as well. To do so, you just need to create key for the API, or JWT token, and provide authentification header to tracing API. So you can test full API cycle, include access rights and etc. If your API is protected, and you do not provide token to Tracing API, you will get unauthorized error.

view details

Matias Insaurralde

commit sha 75a7f46666cb253c2efcb899669b2350379418b9

storage: fallback to Redis default port (#1881)

view details

Ahmet Soormally

commit sha 364f258edeb97e35c62c86ea8204ea2c2046ab0d

improve log processed & listening on domain/path (#1910) When listening on custom domain rather than using listen paths e.g. `foo.com`. The gateway starts, and displays the following log message: ```[Sep 12 16:05:13] INFO main: Processed and listening on: /{rest:.*}``` This makes it impossible to see what APIs are loaded. By adding the `Domain` property to the chain object, we can now provide better logs: ``` [Sep 12 16:08:02] INFO main: Processed and listening on: foo.com/{rest:.*} ```

view details

Matias Insaurralde

commit sha f8446b50aae8c9258fad4bdad667bd6e00e555a4

coprocess: use raw_body when re-building the request object (#1911)

view details

Josh Blakeley

commit sha 1feb5e59a3b8c341c13f17b311c64f11420dd659

if not key in meta info on key found write as empty string (#1919)

view details

Artem Hluvchynskyi

commit sha 84643b27481573852e24ef1238c9649a524e8ef9

Revendored redigocluster for Redis 4+ support (#1920)

view details

Josh Blakeley

commit sha 2124817c469faeb8fefb21e403c71d983ea0fe4f

check for policy change and apply to access rights each request (#1909)

view details

Josh Blakeley

commit sha 2b72c5910951a48b72a8c5b1b004872bf811d8cf

remove forward slash from path join (#1906)

view details

Josh Blakeley

commit sha a0afb8bc07be7fb97827134220689a50ab800b58

respect key logging in reverse proxy (#1905)

view details

Josh Blakeley

commit sha 5dafcffa01fca8a7defe99375df886476fdab799

enable tls renegotiation in reverse proxy and have config option to disable (#1912)

view details

Matias Insaurralde

commit sha c245af2eb3c23e1e93dc165d72b2b5524cb7a9b6

Fix Python memory issues (#1886) * coprocess: fix Python memory issues Added Python refcount macros. Modified the C.free calls to avoid any GC issues. Python_DispatchHook uses memcpy to avoid accessing the internal buffer of the resulting PyObject. * coprocess: modify Lua code to match dispatcher arguments * coprocess: add error results

view details

Leonid Bugaev

commit sha c0b351b99c35ffcac02c84bbd83de69f10ab5c56

Fix #1924 (#1925) Should check if JSVM enabled before loading JS files

view details

dencoded

commit sha e640f4d20d8868ece505fe2243f29153c8a4f4ca

move RPC transport and analytics purger into package (#1923) Another potentially better approach for https://github.com/TykTechnologies/tyk-pump/issues/80 This approach allows to move some re-usable part of RPC without changing RPC server side. if it works then it cancels https://github.com/TykTechnologies/tyk/pull/1922 and https://github.com/TykTechnologies/tyk-sink/pull/51 The idea is to do `rpc.Connect(...)` where we supply connection info, a couple of call backs for emergency mode and map of rpc funcs definitions to add to RPC client dispatcher. Then we can create `rpc.Purger` with all needed fields and run `PurgeLoop` with some ticker.

view details

dencoded

commit sha 55617d918dddbc6b29ba992a5922f91dac19f44a

Switch RPC to TykTechnologies/gorpc (#1926) this is to switch tyk to use our fork with Accept/client ID recent fixes

view details

dencoded

commit sha 227cdf3a4e5cd5944f2c1929f66af36935e51828

EmitErrorEventKv panic fixed (#1927)

view details

Matias Insaurralde

commit sha 4ad0fad73b1739ebd1c7a78a6e5bca2e6e6d54e5

coprocess: fix custom module loader (#1931)

view details

push time in a month

push eventTykTechnologies/tyk

Komal Sukhani

commit sha f87150fc73075ed11a4c98aa98224d07fc5ef6af

Request signing middleware (#2328) The feature is implemented using [Draft 10](https://tools.ietf.org/html/draft-cavage-http-signatures-10) `(request-target)` and all the headers of the request will be used for generating signature string. If request doesn't contain `Date` header, middleware will add one as it is required according to above draft. A new config option `request_signing` is added in API Definition to enable/disable request signing. It has following format ```json "request_signing": { "is_enabled": true, "key": "xxxx", "key_id": "1", "algorithm": "hmac-sha256" } ``` Following algorithms are supported: 1. `hmac-sha1` 2. `hmac-sha256` 3. `hmac-sha384` 4. `hmac-sha512` Fixes #2234

view details

Geofrey Ernest

commit sha e747861a4bc3d977cc9d6ed789ec58c0527bca3e

don't use Request.GetBody (#2434) Decode json directly from r.Body because r.GetBody is nil Fixes #2375 Fixes #2398

view details

Matias Insaurralde

commit sha 27361a3cbde9f0b5ceefa97ff3055e12fcb67a23

coprocess: make the Python custom loader manage only local calls (#2403) For #2402, need to do some more tests before merging.

view details

Leonid Bugaev

commit sha bced1b94a3a1ab70c054fa505d94ecbbd538d1f7

Fix Body Transform with Validate JSON plugin (#2446) We use special `nopCloser` wrapper for body, which is smart enough to automatically rewind the body when it fully read. When we set new body, we need wrap it to noCloser again. Fix https://github.com/TykTechnologies/tyk/issues/2425

view details

Leonid Bugaev

commit sha 7dc2f778d0cfeb2b866ab6177fa44b3e810613a6

Fix formatting of complex error messages (#2449) It was using html/template package for rendering JSON files which was causing HTML based escaping, instead of JS based. For example quotes was escaped as `&#34;` instead of `\"` Fix https://github.com/TykTechnologies/tyk/issues/2448

view details

Furkan Senharputlu

commit sha d3fc15ba8dd2fc8900d34d0a31f1840e2edd01c9

Add mutual, basic and token-based authentication tests for gRPC (#2124) This PR adds the following tests for gRPC: - Mutual Authentication - Basic Authentication - Token Based Authentication extends #2119

view details

Ahmet Soormally

commit sha d80ee5bd86a20a494b110bf31546917e09e1f6f2

(log) updating logrus library (#2454) - As per title - Updated logrus package - updating evalphobia - updating gemnasium graylog hook - updating logrus-logstash-hook - removing unused dependencies - no need to escape json from msg - JSVM log message now contains prefix - and logger no longer leaves erroneous space at the end - Fixes issue where `TextFormatter` shows log as seconds since start, rather than timestamp.

view details

Matias Insaurralde

commit sha 69a6be5a4d0f5eb8e383a5edb1ee6aa5c437f940

Update http2 and grpc vendored packages (#2456) To ensure we incorporate fixes for [CVE-2019-9512](https://nvd.nist.gov/vuln/detail/CVE-2019-9512) (Ping Flood), [CVE-2019-9514](https://nvd.nist.gov/vuln/detail/CVE-2019-9514) (Reset Flood), and [CVE-2019-9515](https://nvd.nist.gov/vuln/detail/CVE-2019-9515) (Settings Flood), described [here](https://github.com/Netflix/security-bulletins/blob/master/advisories/third-party/2019-002.md). 1) `golang.org/x/net/http2` changes are [here](https://github.com/golang/net/commit/74dc4d7220e7acc4e100824340f3e66577424772). 2) `google.golang.org/grpc` was updated to the latest tag that incorporates the fixes: [v1.23.0](https://github.com/grpc/grpc-go/releases/tag/v1.23.0).

view details

Matias Insaurralde

commit sha 28735431cb48ffadc3477196c8db981589ff9b4c

coprocess: use dlpython

view details

Matias Insaurralde

commit sha 85f42ab8cda928a22d6d1f47d21883f758163265

coprocess: reimplement gateway bindings using ctypes

view details

Matias Insaurralde

commit sha 59339c04c4a90ff2843789c0f195f8a4e11b5d22

coprocess: fix logging

view details

push time in a month

push eventmatiasinsaurralde/tyk

Matias Insaurralde

commit sha 69a6be5a4d0f5eb8e383a5edb1ee6aa5c437f940

Update http2 and grpc vendored packages (#2456) To ensure we incorporate fixes for [CVE-2019-9512](https://nvd.nist.gov/vuln/detail/CVE-2019-9512) (Ping Flood), [CVE-2019-9514](https://nvd.nist.gov/vuln/detail/CVE-2019-9514) (Reset Flood), and [CVE-2019-9515](https://nvd.nist.gov/vuln/detail/CVE-2019-9515) (Settings Flood), described [here](https://github.com/Netflix/security-bulletins/blob/master/advisories/third-party/2019-002.md). 1) `golang.org/x/net/http2` changes are [here](https://github.com/golang/net/commit/74dc4d7220e7acc4e100824340f3e66577424772). 2) `google.golang.org/grpc` was updated to the latest tag that incorporates the fixes: [v1.23.0](https://github.com/grpc/grpc-go/releases/tag/v1.23.0).

view details

push time in a month

push eventTykTechnologies/tyk

Matias Insaurralde

commit sha 49eae7018c8465930d6081d0ee7a8c7502263585

coprocess: extend PYTHONPATH

view details

Matias Insaurralde

commit sha b820bc38b7209a2f90f1d34753bc296f9047c5e7

coprocess: reimplement gateway bindings using ctypes

view details

push time in a month

push eventmatiasinsaurralde/dlpython

Matias Insaurralde

commit sha e45ce9c2262509f0a34ee826856c472297ed2bb5

Add helpers

view details

push time in a month

issue commentTykTechnologies/tyk

Python custom loader warns about non-local imports

@buger should we close this? PR was merged 👍

matiasinsaurralde

comment created time in a month

startedfranckjay/HierarchicalFordGoBike

started time in a month

pull request commentTykTechnologies/tyk

Support response middleware when using rich plugins

/gofmt

matiasinsaurralde

comment created time in a month

push eventmatiasinsaurralde/tyk

Matias Insaurralde

commit sha 7c2629cf849cbe6b1c6e4dfa7737105e3fd8da24

coprocess: gofmt and goimports

view details

push time in a month

push eventmatiasinsaurralde/tyk

Matias Insaurralde

commit sha b3cf0242aaf9b800a1521c12dbe5feb3bd00e1f2

coprocess: extend tests to cover response object usage

view details

push time in a month

push eventmatiasinsaurralde/tyk

Matias Insaurralde

commit sha 82e17eac7ff3a93beccff09d7e44e8ea57dfc9a1

coprocess: fix Python log calls

view details

Matias Insaurralde

commit sha d4b9d3af2d3152ac3ce540c3675669652e93aa8a

coprocess: update Python helpers

view details

push time in a month

push eventmatiasinsaurralde/tyk

Matias Insaurralde

commit sha 6164ae6b64eb08e096abdc0787460f64fbd2a0be

coprocess: abort request on response hook errors

view details

push time in a month

push eventmatiasinsaurralde/tyk

Matias Insaurralde

commit sha c3faa0b022f7c6fb692db6282e132735c1fb359d

coprocess: abort request on response hook errors

view details

push time in a month

push eventmatiasinsaurralde/tyk

Matias Insaurralde

commit sha c9d962140a461ab3deabd38257fb6378d1f3891f

coprocess: simplify CoProcessor usage and use debug log for error details

view details

push time in a month

push eventTykTechnologies/tyk

Matias Insaurralde

commit sha cca3e57f2b394ce5e76582a33750b9e78d3be103

gofmt

view details

push time in a month

push eventTykTechnologies/tyk

Matias Insaurralde

commit sha 05b52865cd32794aef3ee8c5d3a9f56c061db72c

vendor: update grpc examples package

view details

push time in a month

PR opened TykTechnologies/tyk

Update http2 and grpc vendored packages

To ensure we incorporate fixes for CVE-2019-9512 (Ping Flood), CVE-2019-9514 (Reset Flood), and CVE-2019-9515 (Settings Flood), described here.

  1. golang.org/x/net/http2 changes are here.

  2. google.golang.org/grpc was updated to the latest tag that incorporates the fixes: v1.23.0.

+10106 -6377

0 comment

120 changed files

pr created time in a month

create barnchTykTechnologies/tyk

branch : http2-fixes

created branch time in a month

push eventmatiasinsaurralde/tyk

Komal Sukhani

commit sha f87150fc73075ed11a4c98aa98224d07fc5ef6af

Request signing middleware (#2328) The feature is implemented using [Draft 10](https://tools.ietf.org/html/draft-cavage-http-signatures-10) `(request-target)` and all the headers of the request will be used for generating signature string. If request doesn't contain `Date` header, middleware will add one as it is required according to above draft. A new config option `request_signing` is added in API Definition to enable/disable request signing. It has following format ```json "request_signing": { "is_enabled": true, "key": "xxxx", "key_id": "1", "algorithm": "hmac-sha256" } ``` Following algorithms are supported: 1. `hmac-sha1` 2. `hmac-sha256` 3. `hmac-sha384` 4. `hmac-sha512` Fixes #2234

view details

Geofrey Ernest

commit sha e747861a4bc3d977cc9d6ed789ec58c0527bca3e

don't use Request.GetBody (#2434) Decode json directly from r.Body because r.GetBody is nil Fixes #2375 Fixes #2398

view details

Matias Insaurralde

commit sha 27361a3cbde9f0b5ceefa97ff3055e12fcb67a23

coprocess: make the Python custom loader manage only local calls (#2403) For #2402, need to do some more tests before merging.

view details

Leonid Bugaev

commit sha bced1b94a3a1ab70c054fa505d94ecbbd538d1f7

Fix Body Transform with Validate JSON plugin (#2446) We use special `nopCloser` wrapper for body, which is smart enough to automatically rewind the body when it fully read. When we set new body, we need wrap it to noCloser again. Fix https://github.com/TykTechnologies/tyk/issues/2425

view details

Leonid Bugaev

commit sha 7dc2f778d0cfeb2b866ab6177fa44b3e810613a6

Fix formatting of complex error messages (#2449) It was using html/template package for rendering JSON files which was causing HTML based escaping, instead of JS based. For example quotes was escaped as `&#34;` instead of `\"` Fix https://github.com/TykTechnologies/tyk/issues/2448

view details

Furkan Senharputlu

commit sha d3fc15ba8dd2fc8900d34d0a31f1840e2edd01c9

Add mutual, basic and token-based authentication tests for gRPC (#2124) This PR adds the following tests for gRPC: - Mutual Authentication - Basic Authentication - Token Based Authentication extends #2119

view details

Ahmet Soormally

commit sha d80ee5bd86a20a494b110bf31546917e09e1f6f2

(log) updating logrus library (#2454) - As per title - Updated logrus package - updating evalphobia - updating gemnasium graylog hook - updating logrus-logstash-hook - removing unused dependencies - no need to escape json from msg - JSVM log message now contains prefix - and logger no longer leaves erroneous space at the end - Fixes issue where `TextFormatter` shows log as seconds since start, rather than timestamp.

view details

push time in a month

startedgnuradio/gnuradio

started time in a month

pull request commentTykTechnologies/tyk

Support response middleware when using rich plugins

Rebased and fixed conflicts

matiasinsaurralde

comment created time in a month

push eventmatiasinsaurralde/tyk

Furkan Senharputlu

commit sha cc0c28e4e93d32801500171673a7b17b74774395

Update AssertResponse to use response body reader again (#2366)

view details

Ahmet Soormally

commit sha 6a560268757ca8a46497fbd93da8219e643f25a9

(cache) reduce cache lookups with singleflight (#2368) Implements singleflight for redis cache-key lookups. This ensures only one execution is in-flight for a given cache-key. If duplicate comes in, the duplicate caller will wait for the original to be returned and receives the same results. Synthetic benchmark with caching enabled before Singleflight: ``` hey -z 60s -c 50 -host httpbin http://gateway.ahmet:8080/httpbin/get Summary: Total: 60.1466 secs Slowest: 0.2900 secs Fastest: 0.0014 secs Average: 0.0116 secs Requests/sec: 4304.2495 Response time histogram: 0.001 [1] | 0.030 [258246] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ 0.059 [538] | 0.088 [1] | 0.117 [2] | 0.146 [0] | 0.175 [0] | 0.203 [1] | 0.232 [8] | 0.261 [49] | 0.290 [40] | Latency distribution: 10% in 0.0074 secs 25% in 0.0089 secs 50% in 0.0109 secs 75% in 0.0134 secs 90% in 0.0164 secs 95% in 0.0186 secs 99% in 0.0241 secs Details (average, fastest, slowest): DNS+dialup: 0.0000 secs, 0.0014 secs, 0.2900 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0045 secs req write: 0.0000 secs, 0.0000 secs, 0.0033 secs resp wait: 0.0115 secs, 0.0013 secs, 0.2810 secs resp read: 0.0001 secs, 0.0000 secs, 0.0045 secs Status code distribution: [200] 258886 responses ``` Synthetic benchmark with caching & singleflight implemented: ``` hey -z 60s -c 50 -host httpbin http://gateway.ahmet:8080/httpbin/get Summary: Total: 60.0035 secs Slowest: 0.3117 secs Fastest: 0.0002 secs Average: 0.0055 secs Requests/sec: 9031.3005 Response time histogram: 0.000 [1] | 0.031 [541361] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ 0.063 [234] | 0.094 [59] | 0.125 [4] | 0.156 [52] | 0.187 [54] | 0.218 [60] | 0.249 [36] | 0.281 [26] | 0.312 [23] | Latency distribution: 10% in 0.0033 secs 25% in 0.0042 secs 50% in 0.0051 secs 75% in 0.0063 secs 90% in 0.0077 secs 95% in 0.0087 secs 99% in 0.0130 secs Details (average, fastest, slowest): DNS+dialup: 0.0000 secs, 0.0002 secs, 0.3117 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0064 secs req write: 0.0000 secs, 0.0000 secs, 0.0124 secs resp wait: 0.0053 secs, 0.0002 secs, 0.3033 secs resp read: 0.0002 secs, 0.0000 secs, 0.1030 secs Status code distribution: [200] 541910 responses ``` This means that for a given cache-key, we can increase performance of a gateway with a cached endpoint by >100% Naturally, if the cache-key is has a large entropy (a more real-life scenario) - e.g. a spread of client IPs / api keys hitting a large variety of endpoints, the performance gain will be less pronounced.

view details

Furkan Senharputlu

commit sha 33e50b442ef2e3259a0ddd47519bd68262c51ae8

Move coprocess tests to the coprocess package and update ci-test.sh (#2356)

view details

Furkan Senharputlu

commit sha 1b6ede792db82fb841ba1cf39f65f1fceffe7250

Add key update test for policies not enforcing acl case (#2373) This PR adds test for the case in which applied policies to key doesn't enforce ACL. Also, this test deprecates `CreateSession` test util function. Tests should use `Test.CreateSession` function instead after this PR.

view details

Furkan Senharputlu

commit sha 1b0a73760f78aead80316c37477b569bd99cb373

Changes for integration (#2376) These changes are necessary to be able to write dashboard-gateway integration tests.

view details

Komal Sukhani

commit sha 96f02ac850dc84eca8bbb57c234d7ba0b0c802fc

Add Secure Headers to Oauth token endpoint (#2378) Added below headers in `/oauth/token` response ``` "X-Content-Type-Options" "X-XSS-Protection" "X-Frame-Options" "Strict-Transport-Security" "Cache-Control" "Pragma" "Expires" ```

view details

Komal Sukhani

commit sha 3015c87f94272004030e46be31063586744ef738

Store token's time of creation (#2388) Added new field `date_created` in SessionState

view details

Leonid Bugaev

commit sha 211b44926addb301a20c6a37f49588bda2dad0c8

Fix SSE streaming (#2377) At the moment it re-use websocket hijacking functionality Fix https://github.com/TykTechnologies/tyk/issues/2386

view details

Furkan Senharputlu

commit sha 6f2c894f361309c648a67a3f1f78c39080014396

Fix race condition that occurs during TestVirtualEndpoint (#2389) Fixes #2374

view details

Leonid Bugaev

commit sha c7f74f6c92f3518dde60d9bd2cd996bdbd7612a3

Fix retrieval of private certificates from Hybrid env (#2391) When certificate is decoded we should update PEM type and remove headers Fix https://github.com/TykTechnologies/tyk/issues/2390

view details

Komal Sukhani

commit sha 36848bfa26bcc2f17d31d076b9a707f4c3d19b7a

Fix webhook (#2372) 1. Added Timeout to HTTP client used for sending a request to webhook 2. Added new flag `useDefaultTemplate` in WebHookHandler 3. Set Content-Type header to "application/json" if default template is used. 4. Handle unsuccessful response code Fixes #2371

view details

Leonid Bugaev

commit sha 1d3d3f57321ffe50a60f98ab7644365611a7b86e

Fix webhook content type for custom templates (#2393) Addition to https://github.com/TykTechnologies/tyk/issues/2371

view details

Leonid Bugaev

commit sha cf0959a832ee2f5e2f3802d53f40ef8d7aa50391

Fix webhook tests

view details

Furkan Senharputlu

commit sha 3432bb89bf925f77855162a0d99d5c7f031e9a00

Fix race on mainRouter (#2394) This race appears in integration test scenarios.

view details

Geofrey Ernest

commit sha 5b372d3cae5a269257dbddf4862df533baf5568d

add opentracing support (#2384) Looks good! This adds initial support for opentracing in the api gateway. ## requirements For this feature to work properly we need an opentracing server implementation. Please see the options available and support status Tracer | Supported -------|-------------- jaeger | :white_check_mark: [zipkin](https://github.com/openzipkin/zipkin-go) | :white_check_mark: [Datadog](https://github.com/DataDog/dd-opentracing-go) | :heavy_multiplication_x: [Instana](https://github.com/instana/go-sensor/blob/master/README.md) | :heavy_multiplication_x: [LightStep](https://github.com/lightstep/lightstep-tracer-go) | :heavy_multiplication_x: # Configuring A new configuration option has been added to the tyk config. ```json "tracing": { "name": "$tracer_name", "enabled": true, "options": {} } ``` `$tracer_name` is the name of the tracer to use. This can be one of the tracers supported (see support table), for example `jaeger` `enabled` if true then opentracing is enabled in the gateway, defaults to false. `options`: arbitrary json object used to initialize a tracer. Each tracer requires different setting to initialize so the shape of the will vary based on the support implementation of the tracer. This is how it looks like, when you enable `jaeger` tracer. ```json "tracing": { "name": "jaeger", "enabled": true, "options": { "serviceName": "tyk-gateway", "disabled": false, "rpc_metrics": false, "tags": null, "sampler": { "type": "const", "param": 1, "samplingServerURL": "", "maxOperations": 0, "samplingRefreshInterval": 0 }, "reporter": { "queueSize": 0, "BufferFlushInterval": 0, "logSpans": true, "localAgentHostPort": "jaeger:6831", "collectorEndpoint": "", "user": "", "password": "" }, "headers": null, "baggage_restrictions": null, "throttler": null } } ``` # What is being traced ? Only time spans, for requests. There is no logging information. # How does it look like ? Imagine you have 2 gateways. proxy for two services `trace_1` and `trace_2` - `gateway_1` : - proxy `/ping` => `trace_1/ping` loaded as "ping api" spec - proxy `/echo` => `trace_1/echo` loaded as "echo api" spec - `gateway_1` : - proxy `/pong` => `trace_2/pong` loaded as "pong api" spec `trace_1/ping` => calls `trace_2/pong` =>calls `trace_1/echo` , and all the calls pass through the respective gateway. tracing trace_1/ping will yield the following trace. <img width="1680" alt="Screen Shot 2019-07-09 at 12 23 01" src="https://user-images.githubusercontent.com/6039952/60877986-88064200-a247-11e9-8f58-6fac1ea5c5cf.png"> graph <img width="1673" alt="Screen Shot 2019-07-09 at 12 23 57" src="https://user-images.githubusercontent.com/6039952/60878025-9f452f80-a247-11e9-8ec3-4df4b47162d3.png"> for zipkin <img width="1645" alt="Screen Shot 2019-07-10 at 17 35 40" src="https://user-images.githubusercontent.com/6039952/60978218-7ac98000-a339-11e9-918a-8ba9fb44e353.png">

view details

Matias Insaurralde

commit sha 998c77a2b4c0a75a239b39e6ae54d28fe04a1778

Fix detailed logging when overriding response with rich plugins (#2387) Based on the virtual endpoint approach. Some tweaks are needed in order to initialize the success handler correctly in `api_loader.go`. After this is ready, we can add it to #2321

view details

Geofrey Ernest

commit sha 810ed3be308b2e30c3d646afb52646bb9b284459

fix panic when failed target lookup (#2396) fixes #2382

view details

Furkan Senharputlu

commit sha 10d82aed98335babb4aa7420aa5df516c78731b0

Add more descriptive message for key failure (#2397) I realized this while fixing a bug in an integration test.

view details

Geofrey Ernest

commit sha ec863d0b213088c414c57f21980484810ffbeaa5

accept oauth credentials as json on /token endpoint (#2398) When Content-Type header is set to application/json . This will decode and adds them to r.Form to be used by osin closes #2375

view details

Geofrey Ernest

commit sha 43e6886ab26c63d7ed5b1481cf0af308c8e4387b

move common header strings to header package (#2399) This weeds out lots of bugs from typing header stings everytime.

view details

push time in a month

startednutansahoo/Time-Series-Analysis-of-GDP

started time in a month

startedpinguino-lim/Multivariable_time_series_model

started time in a month

startedflatlogic/sing-app

started time in a month

startedfacultyai/dash-bootstrap-components

started time in a month

startedPold87/academic-keyword-occurrence

started time in a month

create barnchTykTechnologies/tyk

branch : dlpython

created branch time in a month

push eventmatiasinsaurralde/dlpython

Matias Insaurralde

commit sha 8aabc5ae447a0b399a37cb73a631a598732d2b09

Update binding

view details

Matias Insaurralde

commit sha 1046c53fb0c587ca4fd0f29b3aaf9dacab178fa7

Check for errors when mapping calls

view details

push time in a month

push eventmatiasinsaurralde/tyk

Matias Insaurralde

commit sha 011d1b24f547ebe7edd38dc87e4503616a7e2667

coprocess: use dlpython

view details

push time in a month

pull request commentTykTechnologies/tyk

Make the Python custom loader handle only local calls

@buger This is ready, waiting for reviews

matiasinsaurralde

comment created time in 2 months

startedocamposj/Ara_Project

started time in 2 months

push eventmatiasinsaurralde/tyk

Matias Insaurralde

commit sha 92ea0927af940b9ba6b627fe599bc20c1c9bfa1a

temp

view details

push time in 2 months

push eventmatiasinsaurralde/tyk-grpc-go-basicauth-jwt

Matias Insaurralde

commit sha 6d093a561cc27de3494c59a8969b10c4d69c5be6

Add syntax highlighting

view details

push time in 2 months

startedTykTechnologies/tyk-grpc-go-basicauth-jwt

started time in 2 months

push eventmatiasinsaurralde/stp-dataset

Matias Insaurralde

commit sha 8739fbcc72ec1e263ff30be93a847c456f76ff2a

Add dataset

view details

push time in 2 months

push eventmatiasinsaurralde/stp-dataset

Matias Insaurralde

commit sha 70bca3d451e9af15c75ceea9c45141341e9433c0

Merge sectors field

view details

push time in 2 months

push eventmatiasinsaurralde/stp-dataset

Matias Insaurralde

commit sha 9361b7eb855d273a01e08b03b815b81278efc71f

Extract STP geographic coordinates

view details

Matias Insaurralde

commit sha 7b614adf4c76befd39b285d870d85104fcdca692

Add JSON data from IASP

view details

push time in 2 months

push eventmatiasinsaurralde/stp-dataset

Matias Insaurralde

commit sha b7aa65753a09affbc6393540b8b7ae30c09d002d

Use prefix for specific IASP fields

view details

push time in 2 months

startedmatiasinsaurralde/stp-dataset

started time in 2 months

push eventmatiasinsaurralde/stp-dataset

Matias Insaurralde

commit sha 4a1abffe9597403083f167451fd62b03acc12cff

Update README

view details

push time in 2 months

push eventmatiasinsaurralde/stp-dataset

Matias Insaurralde

commit sha e7a5147913b877ce5f0fec19a61dd882c548f3aa

Add script

view details

push time in 2 months

create barnchmatiasinsaurralde/stp-dataset

branch : master

created branch time in 2 months

created repositorymatiasinsaurralde/stp-dataset

created time in 2 months

create barnchmatiasinsaurralde/tyk

branch : dlpython2

created branch time in 2 months

push eventmatiasinsaurralde/tyk

Matias Insaurralde

commit sha bb1fbb6f814a9a06a05a3b43999775c6bdc9f0ed

mw_js_plugin: extend ReturnOverrides HTTP code check to allow redirects (#2429) (#2430) Fix for #2429, with this patch the following code is able to trigger a redirect: ```js var sampleMiddleware = new TykJS.TykMiddleware.NewMiddleware({}); sampleMiddleware.NewProcessRequest(function(request, session, spec) { console.log("sampleMiddleware is called") request.ReturnOverrides.ResponseCode = 301 request.ReturnOverrides.ResponseHeaders = { "Location": "http://anotherurl.com/" } return sampleMiddleware.ReturnData(request, session.meta_data); }); ```

view details

Furkan Senharputlu

commit sha fc90b2dfbf549e633bd568ebbf39015a34065cd8

Fix vendor (#2437)

view details

Furkan Senharputlu

commit sha 486cc8b2ec449aaed14921e136ec484c4b5d03e7

Guard NodeID to handle race condition (#2436) Fixes #2435

view details

Ahmet Soormally

commit sha 12fa088e5d21380a3cfd26865d588192f8eca9d9

perf: optimising stripping bearer token (#2433) Should speed up JWT & Auth Token a little bit. ``` benchcmp old.txt new.txt benchmark old ns/op new ns/op delta BenchmarkStripBearer-4 165 38.4 -76.73% benchmark old allocs new allocs delta BenchmarkStripBearer-4 2 1 -50.00% benchmark old bytes new bytes delta BenchmarkStripBearer-4 96 8 -91.67% ```

view details

push time in 2 months

push eventmatiasinsaurralde/tyk-pump

Leonid Bugaev

commit sha 73bb2801e001e99aa64247b2a31be46dfcdee5bf

Bump version

view details

dencoded

commit sha b7eaa754f5887292dfcb617deada0de472cc4969

Hybrid pump (#83) added changes for https://github.com/TykTechnologies/tyk-pump/issues/80 the same idea - just specify new `"hybrid"` pump in config and will start purging analytics to RPC stack.

view details

Leonid Bugaev

commit sha ec1adc48bb6c65b808b6352ebd57beaef56cf32d

Add prometheus support (#91)

view details

Leonid Bugaev

commit sha 667895a84f80480fd0f615f14626e677294f2db6

Fix typos in readme and formatting

view details

Ahmet Soormally

commit sha 83cc4723b4a56cfb358f910169577cea462e0e5f

fixing example and log prefix (#95)

view details

Leonid Bugaev

commit sha 96ea83398ce1a824eb670011208ff0740a9e0358

Add support for multiple pumps of the same type (#97) Add new "type" field which is used to determine pump type. It falls back to using "key" if "type" not specified So backward compatible with previous configs Now you can define multiple pumps like this: ``` "csv": { "type": "csv", "meta": { "csv_dir": "./" } }, "csv_alt": { "type": "csv", "meta": { "csv_dir": "./" } }, ```

view details

Leonid Bugaev

commit sha 13507e0e6ca6717958eb85673deaf48eb8f27c39

Support ES6 (#78) "version" field now support "6" Fix https://github.com/TykTechnologies/tyk-pump/issues/68

view details

Leonid Bugaev

commit sha 07cc82d3d71970138d625fd470e62545d65bc733

Added support for redis timeouts (#101) Added new "storage.timeout" option Fix https://github.com/TykTechnologies/tyk/issues/2165

view details

Ido Halevi

commit sha 8eefcc2f2488145926b21bc52305da0a5be3f196

Logzio pump (#103) Add Logz.io as a pump

view details

Ahmet Soormally

commit sha 55d108d74f9fa8a57b5de18219cc49423086391c

adding logzio dependencies to vendor (#107) Fix vendoring

view details

furkansenharputlu

commit sha 16a3e5c056577cd2f62b817193257b228f41bb12

Add Host field to AnalyticsRecord

view details

Furkan Senharputlu

commit sha e2c9e0a084f8125fecd2830ac650b619cc883908

Merge pull request #108 from furkansenharputlu/host Add Host field to AnalyticsRecord

view details

Leonid Bugaev

commit sha eb784168238c39ea830f8fbc1398d77e680eeba3

Update version.go

view details

dencoded

commit sha a3ee54a35d12f73b1b7c8eadf1cfaa173cb92793

aggregate logic moved to package, call PurgeAnalyticsDataAggregated added (#110) added changes for https://github.com/TykTechnologies/tyk-pump/issues/105 - new method `PurgeAnalyticsDataAggregated` is called if `"aggregated" true` is specified in pump config - moved aggregate logic to `tyk-pump/analytics` package

view details

Furkan Senharputlu

commit sha 89f197fda6cc82a8ea1620ce4c3d1a1751a92830

Fix vendor (#109)

view details

Ahmet Soormally

commit sha 4006eafcfc7f2d4c86a35159824414b6cc8b8b5f

DogStatsd pump (#94) Pump for sending request analytics to datadog agent Fix https://github.com/TykTechnologies/tyk-pump/issues/104 Requires documentation

view details

Leonid Bugaev

commit sha 576be76aa5803f0279ee3ee32ad8417700aed999

Update elasticsearch.go

view details

Leonid Bugaev

commit sha b90b07443fc7278f25dd43cfcfd078e45f865f88

Update README.md

view details

Alok G Singh

commit sha 400117a59abde7457557754bb4c8ab3c553f33dd

Signing rpms using gpg2 and the agent (#116) This allows us to use the official golang 1.12 images rather than having to install it by hand on jessie.

view details

Sedky Abou-Shamalah

commit sha 46dafd9cdeed43ab396c8f073e397a05aadd01d4

Add section on Mongo-aggregate plugin (#119) * Update README.md * Update README.md

view details

push time in 2 months

more