profile
viewpoint
Johannes Wienke languitar Plan.One Germany http://www.semipol.de If I am not programming or doing stuff with robots, I am a photographer: https://www.johanneswienke.de

languitar/autosuspend 35

A daemon to automatically suspend and wake up a system

languitar/config-vim 16

my vim/neovim config

languitar/broadcast-logging 1

A Python logging system handler which broadcasts log messages and a matching receiver executable.

languitar/academic-keyword-occurrence 0

Extracts the historic word occurrence of a search term in academic papers

languitar/aclidswitch 0

Simple Power Management tool to define custom lid, brightness and low battery actions depending on the laptop's AC state

languitar/ale 0

Asynchronous Lint Engine

languitar/Android-Password-Store 0

Android application compatible with ZX2C4's Pass command line application

languitar/android_kernel_lge_hammerhead 0

Kernel source code for LGE Hammerhead (Google Nexus 5)

languitar/ansible-role-dependency-issue 0

demonstrates an issue with role dependencies and parameters with same names

PR opened zalando/connexion

Pass through Werkzeug exception headers to problem

Some Werkzeug exceptions such as Unauthorized define headers to present to the client. Pass through these headers to the problem render method to actually deliver those headers to the client.

Might potentially fix #1234

+4 -2

0 comment

1 changed file

pr created time in 3 days

create barnchlanguitar/connexion

branch : pass-through-werkzeug-exception-headers

created branch time in 3 days

fork languitar/connexion

Swagger/OpenAPI First framework for Python on top of Flask with automatic endpoint validation & OAuth2 support

https://connexion.readthedocs.io/en/latest/

fork in 3 days

issue commentkoxudaxi/datamodel-code-generator

Nested additionalProperties result in empty value type being generated

Thanks again for the very fast fixes!

languitar

comment created time in 3 days

issue commentkoxudaxi/datamodel-code-generator

Provide default value also for required fields

Did you want an option like --use-default ? This option sets default value on required field.

If there's no chance to always set the defaults, that would be ok for me.

languitar

comment created time in 4 days

issue commentkoxudaxi/datamodel-code-generator

Nested additionalProperties result in empty value type being generated

Interestingly, this doesn't happen if additionalProperties is used on the top-level. This result then in:

class Test(BaseModel):
    pass

    class Config:
        extra = Extra.allow
languitar

comment created time in 4 days

issue openedkoxudaxi/datamodel-code-generator

Nested additionalProperties result in empty value type being generated

Describe the bug

Nested additionalProperties with specified type result in empty value type being generated.

To Reproduce

Example schema:

openapi: "3.0.0"
info:
  title: Sample API
  description: API description in Markdown.
  version: 1.0.0
servers: []
paths: {}
components:
  schemas:
    test:
      type: object
      properties:
        failing:
          type: object
          additionalProperties:
            type: string
          default: {}

Used commandline:

$ datamodel-codegen --input openapi.yaml --output test.py

Resulting generated code:

from __future__ import annotations

from typing import Dict, Optional

from pydantic import BaseModel


class Failing(BaseModel):
    pass


class Test(BaseModel):
    failing: Optional[Dict[str, Failing]] = None

The following error occurs when trying to use these generated classes:

print(Test(failing={"foo": "bar"}).to_json())
Traceback (most recent call last):
  File "test.py", line 20, in <module>
    print(Test(failing={"foo": "bar"}).to_json())
  File "pydantic/main.py", line 346, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for Test
failing -> foo
  value is not a valid dict (type=type_error.dict)

Expected behavior

There is no type Failing or it is correctly set to be of value type str. Right now, data cannot be generated with this output.

Version:

  • OS: Linux
  • Python Verison: 3.8
  • datamodel-code-generator Version: 0.5.34

created time in 4 days

issue openedkoxudaxi/datamodel-code-generator

Provide default value also for required fields

Is your feature request related to a problem? Please describe.

Assume the following specifications:

openapi: "3.0.0"
info:
  title: Sample API
  description: API description in Markdown.
  version: 1.0.0
servers: []
paths: {}
components:
  schemas:
    test:
      type: object
      properties:
        broken:
          type: string
          default: "a string"
          description: "dummy"
      required:
        - broken

Right now, the generated Field broken will be:

broken: str = Field(..., description='dummy')

This is in line with the OpenAPI specification but especially for the purpose of generating data this is pretty annoying, because you always have to provide default values that are already known from the specification.

Describe the solution you'd like

In case of required + default, the default should still be spliced into the field. I don't see how this could harm apart from providing convenience.

Describe alternatives you've considered

If the fear is that error situations are missed when parsing inputs, an optional command line flag could be used to realize this.

Additional context Add any other context or screenshots about the feature request here.

created time in 4 days

pull request commentzalando/connexion

Allow nesting of path resources in RestyResolver

Any plans for a new release including these changes? We'd like to use them.

drewsonne

comment created time in 5 days

issue commentcookiecutter/cookiecutter

[user config] prompt_for_config does not iterate through sub dictionaries

If dictionary variables are moved into something as proposed in this issue, the use case of using them for variable-length input from users is removed again. I want users to provide a list of options, which already isn't possible with list values. If dictionaries are also meant for choices, variable input is again lacking.

lbrack

comment created time in 6 days

issue commentcruft/cruft

Update procedure for changed variables

What I am talking about are the answers (inputs) that you provide to cookiecutter, which are the entries defined in cookiecutter.json.

Imagine I have created a new project from a template and initially selected a value foo for my input. Later, I change my mind about that value and want it to be bar. Right now, cruft update -i will only ask again for the inputs in case the template itself has changed. However, I would like to be able to change an input that way, even if the template didn't change at all. In the most extreme case, it might even be a good idea to have something like a --force flag, because an input might be a URL that is processed by a hook in the template and I want to rerun that hook with updated contents from the same URL.

languitar

comment created time in 7 days

issue commentpython-poetry/poetry

Support multiple private repositories

Oh yes thanks. I missed the double brackets when reading the examples

marmida

comment created time in 7 days

issue commentpython-poetry/poetry

Support multiple private repositories

How would that work with the repository specification inside the pyproject.toml in tool.poetry.source?

marmida

comment created time in 9 days

issue commentkoxudaxi/datamodel-code-generator

Option to disable timestamp

Thanks, works as expected

languitar

comment created time in 9 days

issue commentnedbat/coveragepy

Missing TOTAL line in report when omitting all but one file

Just to add one more hint to this: Gitlab only supports Regular Expressions for parsing the generated output. Without the TOTAL, creating a working regular expression is pretty hard.

Hisol

comment created time in 9 days

issue commentzalando/connexion

[Question] Is there a prescribed way to generate response model from openapi spec?

https://github.com/koxudaxi/datamodel-code-generator is not perfect but quite usable now for this task. However, using pydantic objects directly for responses is not easily possible right now.

arajkumar

comment created time in 11 days

issue openedkoxudaxi/datamodel-code-generator

Option to disable timestamp

Is your feature request related to a problem? Please describe.

Please add an option to disable the timestamp comment in generated files. This conflicts with the idea of reproducible builds: https://reproducible-builds.org/docs/timestamps/. Given the same specification the same output should be generated.

Describe the solution you'd like

An additional command line switch would be ok for us.

Describe alternatives you've considered

I don't see many alternatives that are not a brittle hack.

created time in 11 days

issue commentcruft/cruft

Update procedure for changed variables

Actually, this would already suffice:

        if not cookiecutter_input and is_project_updated(repo, cruft_state["commit"], last_commit, strict):
            typer.secho(
                "Nothing to do, project's cruft is already up to date!", fg=typer.colors.GREEN
languitar

comment created time in 12 days

issue commentcopier-org/copier

When copier fails, it should not delete the destination path when it pre-exists

This is extremely unexpected when updating an existing project. I don't think this ever makes sense for updates. Only for the initial creation.

matt2000

comment created time in 12 days

issue openedcopier-org/copier

no-diff command line flag not used in 5.1.0

The --no-diff command line flag is not obeyed in that version. In make_config it ends up in kwargs, but is never read from there.

created time in 12 days

issue openedcruft/cruft

Update procedure for changed variables

What is the intended way to update a project in case of a changed variable in .cruft.json? Looks like a feature for this is missing because only the git commit is checked, not changes in this file.

created time in 12 days

push eventlanguitar/fastapi-code-generator

Johannes Wienke

commit sha 4c29be8abdde35cf5348a7b25e7ab1ff82df2c7d

Fix license field in pyproject.toml Lower-case version is expected by modern poetry versions.

view details

push time in 13 days

fork languitar/fastapi-code-generator

This code generator creates FastAPI app from an openapi file.

fork in 13 days

issue openedkoxudaxi/fastapi-code-generator

Arrays always assumed to contain a $ref

The following spec currently results in a generation error:

openapi: 3.0.0
info:
  title: pet-finder
  version: '1.0'
servers:
  - url: 'http://localhost:3000'
paths:
  /find/:
    get:
      summary: find
      tags: []
      responses:
        '200':
          description: OK
          content:
            application/json:
              schema:
                type: array
                items:
                  type: integer
              examples:
                example-1:
                  value:
                    - 0
                    - 1
                    - 3
      operationId: get-find
components:
  schemas: {}

Error message:

❯ poetry run fastapi-codegen -i ../petfinder/reference/pet-finder.v1.yaml -o /tmp/petfinder
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/languitar/src/fastapi-code-generator/.venv/lib/python3.8/site-packages/typer/main.py", line 213, in __call__
    return get_command(self)()
  File "/home/languitar/src/fastapi-code-generator/.venv/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/languitar/src/fastapi-code-generator/.venv/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/languitar/src/fastapi-code-generator/.venv/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/languitar/src/fastapi-code-generator/.venv/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/languitar/src/fastapi-code-generator/.venv/lib/python3.8/site-packages/typer/main.py", line 496, in wrapper
    return callback(**use_params)  # type: ignore
  File "/home/languitar/src/fastapi-code-generator/fastapi_code_generator/__main__.py", line 26, in main
    return generate_code(input_name, input_text, output_dir, template_dir)
  File "/home/languitar/src/fastapi-code-generator/fastapi_code_generator/__main__.py", line 37, in generate_code
    parsed_object: ParsedObject = parser.parse()
  File "/home/languitar/src/fastapi-code-generator/fastapi_code_generator/parser.py", line 391, in parse
    return self.parse_paths(openapi)
  File "/home/languitar/src/fastapi-code-generator/fastapi_code_generator/parser.py", line 400, in parse_paths
    return ParsedObject(
  File "/home/languitar/src/fastapi-code-generator/fastapi_code_generator/parser.py", line 375, in __init__
    operation.response
  File "/usr/lib/python3.8/functools.py", line 967, in __get__
    val = self.func(instance)
  File "/home/languitar/src/fastapi-code-generator/fastapi_code_generator/parser.py", line 252, in response
    type_ = f'List[{schema.items.ref_object_name}]'
  File "/home/languitar/src/fastapi-code-generator/.venv/lib/python3.8/site-packages/datamodel_code_generator/parser/jsonschema.py", line 115, in ref_object_name
    return self.ref.rsplit('/', 1)[-1]  # type: ignore
AttributeError: 'NoneType' object has no attribute 'rsplit'

created time in 13 days

issue commenttableau/server-client-python

NotSignedInError despite using context manager

Unfortunately, I haven't seen this happen since weeks. So I doubt I'll be able to reproduce this reliably.

languitar

comment created time in 14 days

push eventlanguitar/config-vim

Johannes Wienke

commit sha 56d5a41cfc8df9ec0565a1ae5f65e87415d7b574

Add helm syntax highlighting

view details

push time in 16 days

issue commentmongodb/terraform-provider-mongodbatlas

Can't create alerts with using many event_type

@rafafalves I hope you have changed the API token after this report ;-)

rafafalves

comment created time in 17 days

startedkoxudaxi/datamodel-code-generator

started time in 18 days

issue commentkoxudaxi/datamodel-code-generator

additionalProperties always result in Any Python type

This already happens without a type reference. Setting type: string exhibits the same behavior.

languitar

comment created time in 18 days

issue openedkoxudaxi/datamodel-code-generator

Type references in additionalProperties result in Any Python type

Describe the bug

When using an object with additionalProperties and an item type, the resulting code doesn't reference the generated item type.

To Reproduce

Example schema:

openapi: "3.0.0"
info:
  title: Sample API
  description: API description in Markdown.
  version: 1.0.0
servers: []
paths: {}
components:
  schemas:
    test:
      type: object
      properties:
        broken:
          type: object
          additionalProperties:
            $ref: '#/components/schemas/broken'
    broken:
      type: object
      properties:
        foo:
          type: string
        bar:
          type: integer

Results in the following generated file:

# generated by datamodel-codegen:
#   filename:  openapi.yaml
#   timestamp: 2020-09-09T10:17:59+00:00

from __future__ import annotations

from typing import Any, Dict, Optional

from pydantic import BaseModel


class Test(BaseModel):
    broken: Optional[Dict[str, Any]] = None


class Broken(BaseModel):
    foo: Optional[str] = None
    bar: Optional[int] = None

Used commandline:

$ datamodel-codegen --input openapi.yaml --output broken.py

Expected behavior The definition for broken should be:

broken: Optional[Dict[str, Broken]] = None

Version:

  • OS: Linux
  • Python Verison: 3.8.5
  • datamodel-code-generator Version: 0.5.30

created time in 18 days

issue openedtiangolo/fastapi

Select response model specified in responses based on status code

First check

  • [x] I added a very descriptive title to this issue.
  • [x] I used the GitHub search to find a similar issue and didn't find it.
  • [x] I searched the FastAPI documentation, with the integrated search.
  • [x] I already searched in Google "How to X in FastAPI" and didn't find any information.
  • [x] I already read and followed all the tutorial in the docs and didn't find an answer.
  • [x] I already checked if it is not related to FastAPI but to Pydantic.
  • [x] I already checked if it is not related to FastAPI but to Swagger UI.
  • [x] I already checked if it is not related to FastAPI but to ReDoc.
  • [x] After submitting this, I commit to:
    • Read open issues with questions until I find 2 issues where I can help someone and add a comment to help there.
    • Or, I already hit the "watch" button in this repository to receive notifications and I commit to help at least 2 people that ask questions in the future.
    • Implement a Pull Request for a confirmed bug

Example

Here's a self-contained minimal, reproducible, example with my use case:

from fastapi import FastAPI, Response
from pydantic import BaseModel

api = FastAPI()


class DefaultModel(BaseModel):
    default: str = "yes"


class OtherModel(BaseModel):
    something: str = "something"
    other: str = "other"


@api.get("/test", response_model=DefaultModel, responses={503: {"model": OtherModel}})
def get(response: Response):
    response.status_code = 503
    return OtherModel()

Description

In contradiction to the documentation for the 503 status code, the sample code will result in a reply {"default": "yes"} without any validation errors.

It would be nice of fastapi could select the correct model depending on the status code declared in the response object.

The solution you would like

The code should result in an OtherModel being serialized instead of the default one based on the selected status code.

Describe alternatives you've considered

Using JSONResponse manually is tedious and error prone.

Environment

  • OS: [e.g. Linux / Windows / macOS]: linux
  • FastAPI Version [e.g. 0.3.0]: 0.61.1
  • Python version: 3.8.5

created time in 20 days

pull request commentencode/uvicorn

Fixes bug where --log-config disables uvicorn loggers

Any chance to push out a release with this fix included?

petermorrow

comment created time in 20 days

create barnchlanguitar/xrays

branch : feature/complexity-per-revision

created branch time in 21 days

delete branch languitar/xrays

delete branch : feature/dash

delete time in 21 days

push eventlanguitar/xrays

Johannes Wienke

commit sha b5d59b7f1fc92843eeff041b7f8302576a49c5a4

TEMP use dash for creating a dashboard

view details

Johannes Wienke

commit sha 0cf504cf8cf27286fd0dfd699533a0bb67b4d4fe

Revert to Dash

view details

Johannes Wienke

commit sha cc08757d7dc09cc45b7107b23d1ce6c6f1b60653

Split into separate modules

view details

Johannes Wienke

commit sha df3585ee8cbbdff0e597bf7d4382fa5b0d555227

Use tabs

view details

Johannes Wienke

commit sha a48b5ade3ddc6a004c1a99464e161924165f100f

Merge branch 'feature/dash' into master

view details

push time in 21 days

delete branch languitar/pass-git-helper

delete branch : bugfix/lint

delete time in 21 days

push eventlanguitar/pass-git-helper

Johannes Wienke

commit sha a59e738d1c820aa58b4e2bb08aa4264a3a126253

Pin linters and upgrade black Adapt to new black style and avoid future unintended changes through pinned versions.

view details

Johannes Wienke

commit sha 0180749f840a8e2a2cbb4fbf1d8172869926e289

Merge pull request #20 from languitar/bugfix/lint Pin linters and upgrade black

view details

push time in 21 days

PR merged languitar/pass-git-helper

Pin linters and upgrade black bug enhancement

Adapt to new black style and avoid future unintended changes through pinned versions.

+50 -24

1 comment

2 changed files

languitar

pr closed time in 21 days

PR opened languitar/pass-git-helper

Pin linters and upgrade black bug enhancement

Adapt to new black style and avoid future unintended changes through pinned versions.

+50 -24

0 comment

2 changed files

pr created time in 21 days

create barnchlanguitar/pass-git-helper

branch : bugfix/lint

created branch time in 21 days

delete branch languitar/autosuspend

delete branch : bugfix/lint

delete time in 21 days

push eventlanguitar/autosuspend

Johannes Wienke

commit sha 21e0be903c5558032eb29a91bc2f6399f05b10c5

Fix linting with new black version Pin the linting environment to exactly this black version.

view details

Johannes Wienke

commit sha 136437a36be0a36d362bf74fbde064229cf63832

Merge pull request #87 from languitar/bugfix/lint Fix linting with new black version

view details

push time in 21 days

PR merged languitar/autosuspend

Fix linting with new black version bug enhancement

Pin the linting environment to exactly this black version.

+85 -29

1 comment

10 changed files

languitar

pr closed time in 21 days

PR opened languitar/autosuspend

Fix linting with new black version bug enhancement

Pin the linting environment to exactly this black version.

+85 -29

0 comment

10 changed files

pr created time in 21 days

create barnchlanguitar/autosuspend

branch : bugfix/lint

created branch time in 21 days

issue commentkoxudaxi/datamodel-code-generator

Pass through example strings

Thanks! that was quick!

languitar

comment created time in 22 days

issue openedkoxudaxi/datamodel-code-generator

Pass through example strings

Is your feature request related to a problem? Please describe.

As a validation step for using this tool I have taken an existing API built with fastapi and pydantic models and applied the code generator on the openapi specification provided by openapi. In the generated models the example is missing. As we want to take parts of the generated models also as output models again, it would be nice if those examples could also be passed into the pydantic models.

Describe the solution you'd like

In case the openapi specification contains a custom example for a field, pass this example to the example argument of the pydantic Field.

Describe alternatives you've considered

I don't see an alternative.

created time in 23 days

issue commentmongodb/terraform-provider-mongodbatlas

inherited_roles are not correctly removed from custom_db_roles

We will try our best within the next few days.

languitar

comment created time in a month

issue commentlanguitar/autosuspend

Conditions tester

There is an option to the daemon to iterate all checks, even if one of them indicates activity: https://autosuspend.readthedocs.io/en/3.0/options.html#cmdoption-autosuspend-daemon-a You can use that for developing your config. For testing, you also don't need to use the systemd service. You can just start autosuspend manually on a root shell.

BLLuis

comment created time in a month

issue commenttableau/server-client-python

NotSignedInError despite using context manager

Its not fixed, but happens a lot less frequently now.

Since this works most of the time, I don't see how site_id should help with just a single site?

languitar

comment created time in a month

issue openedDeanWay/fastapi-versioning

dependency_overrides cannot be used in tests as expected, documentatin lacking

Describe the bug

Without fastapi-versioning, dependency_overrides can easily be specified on the app contained in the TestClient a la (pytest fixture like pseudo code):

before = client.app.dependency_overrides
client.app.dependency_overrides = dict(overrides)
try:
    yield
finally:
    client.app.dependency_overrides = before

With the internal use of sub application mounts, this doesn't work anymore and the override never reaches one of the versioned sub applications.

To Reproduce Steps to reproduce the behavior:

from fastapi import APIRouter, Depends, FastAPI
from fastapi.testclient import TestClient
from fastapi_versioning import versioned_api_route, VersionedFastAPI

app = FastAPI(title="test-app")
router = APIRouter(route_class=versioned_api_route(1, 0))

def dependency() -> str:
    return "original"

@router.get("/test")
def get_test(dep: str = Depends(dependency)) -> str:
    return dep

app.include_router(router)
app = VersionedFastAPI(app)


def test_this() -> None:
    client = TestClient(app)
    client.app.dependency_overrides = {dependency: lambda: "patched"}
    assert client.get("/v1_0/test").json() == "patched"

This test will fail

Expected behavior

It should at least be documented how to access the correct sub application for providing the dependency_overrides.

created time in a month

pull request commenti8enn/pydantic-odm

Summary grammar fix

No problem. It's included now.

Just as an alternative: for the languages I speak DeepL gives much better results.

languitar

comment created time in a month

push eventlanguitar/pydantic-odm

Johannes Wienke

commit sha 8d4d57f7a9a0b307444fe4672795039eb012745d

Also fix the summary in setup.py

view details

push time in a month

PR opened i8enn/pydantic-odm

Summary grammar fix
+2 -2

0 comment

2 changed files

pr created time in a month

push eventlanguitar/pydantic-odm

Johannes Wienke

commit sha 36f37b17ef563f3c7c96f1c39dd0cef3e444ac12

Also fix grammar in pyproject.toml

view details

push time in a month

create barnchlanguitar/pydantic-odm

branch : summary-grammar-fix

created branch time in a month

fork languitar/pydantic-odm

Small async ODM for MongoDB based in Motor and Pydantic library

fork in a month

issue commenttiangolo/fastapi

[QUESTION] Passing additional config args like "scopes" to the callable in a Security function.

If it's configurations per dependency, then what you need is a dependency class: https://fastapi.tiangolo.com/tutorial/dependencies/classes-as-dependencies/

I tried to realize this pattern for setting the openIdConnectUrl in an OpenIdConnect instance by acquiring the URL from injected settings. But I can't make this work in combination with the Security class. @tiangolo do you have an implementation hint how to realize this. Some general documentation on how to realize this would also be very helpful.

jjwwiodyne

comment created time in a month

issue commentmongodb/terraform-provider-mongodbatlas

custom_db_roles cannot be created with only inherited roles

Thanks for the fix. Are there any plans to generate a release including this fix?

languitar

comment created time in a month

issue commentvim-test/vim-test

Support java integration tests (maven failsafe)

But this needs a new command which I named as :IntegrationTest for now. Until other languages implement this kind of behaviour it would require manual configuration by adding something like this to your .vimrc:

command! -nargs=* -bar IntegrationTest call test#run('integration', split(<q-args>))

Another option could be to define a cascade of commands to execute, first unit and then integration tests. I think the original reason for my request was specifically to support TestNearest. That would need some dispatching then, which of the commands is relevant here.

languitar

comment created time in a month

issue commenttiangolo/fastapi

Search feature on documentation website does not work

Same issue here: interestingly, the documentation root always seems to work (at least after a refresh without cache) but other pages mostly not. Refresh without cache sometimes makes it work on other pages, but often not.

ansidev

comment created time in a month

issue openedDeanWay/fastapi-versioning

Transformation-based versioning

Is your feature request related to a problem? Please describe.

Maintaining multiple API versions explicitly in the code base results in higher maintenance costs with each additional version that is kept alive. This blog post by strip explains an interesting idea of countering this fact by representing versions via transformations to the responses instead, such that the code base, apart from declaring the transformations, only has to deal with the most recent API versions and all other versions are implicitly formed via the transformations.

Describe the solution you'd like

It would be cool of fastapi-versioning (or potentially another module) would realize a transformation-based approach to recude the long-term costs of maintaining multiple versions in parallel.

Additional context

I could find an implementation of this pattern for the django rest framework: https://github.com/mrhwick/django-rest-framework-version-transforms/tree/dev/tests. This could serve as an inspiration.

created time in a month

Pull request review commentmongodb/terraform-provider-mongodbatlas

Fix #279 #280

 func testAccMongoDBAtlasCustomDBRolesConfigBasic(projectID, roleName, action, da }  func testAccMongoDBAtlasCustomDBRolesConfigWithInheritedRoles(projectID string, inheritedRole []matlas.CustomDBRole, testRole *matlas.CustomDBRole) string {-	log.Printf("LOG___ inheritedRole: %#+v\n", inheritedRole[1])

Why this change?

PacoDw

comment created time in a month

Pull request review commentmongodb/terraform-provider-mongodbatlas

Fix #279 #280

 func TestAccResourceMongoDBAtlasCustomDBRoles_WithInheritedRoles(t *testing.T) {  	inheritRole := []matlas.CustomDBRole{ 		{-			RoleName: fmt.Sprintf("a_test-acc-INHERITED_ROLE-%s", acctest.RandString(5)),+			RoleName: fmt.Sprintf("test-acc-INHERITED_ROLE-%s", acctest.RandString(5)),

Why is this change necessary?

PacoDw

comment created time in a month

Pull request review commentmongodb/terraform-provider-mongodbatlas

Fix #279 #280

 func flattenActionResources(resources []matlas.Resource) []map[string]interface{ }  func expandInheritedRoles(d *schema.ResourceData) []matlas.InheritedRole {-	var inheritedRoles []matlas.InheritedRole+	vIR := d.Get("inherited_roles").(*schema.Set).List()

I would propose to use speaking variable names.

PacoDw

comment created time in a month

issue commentpandas-dev/pandas

ENH: Support downcasting of nullable dtypes in to_numeric

Interestingly, this worked with pandas < version 1.0. I recently upgraded our existing code base from 0.2x to 1.1 and now the existing and unmodified code fails with exactly this error. So, this is a regression:

Before 1.0:

In [7]: pd.to_numeric(pd.Series([42], dtype="Int64"), downcast="integer")
Out[7]: 
0    42
dtype: int8

In [8]: pd.show_versions()

INSTALLED VERSIONS
------------------
commit           : None
python           : 3.8.5.final.0
python-bits      : 64
OS               : Linux
OS-release       : 5.7.12-arch1-1
machine          : x86_64
processor        : 
byteorder        : little
LC_ALL           : None
LANG             : en_US.UTF-8
LOCALE           : en_US.UTF-8

pandas           : 0.25.3
numpy            : 1.19.1
pytz             : 2020.1
dateutil         : 2.8.1
pip              : 20.1.1
setuptools       : 47.1.1
Cython           : None
pytest           : None
hypothesis       : None
sphinx           : None
blosc            : None
feather          : None
xlsxwriter       : None
lxml.etree       : None
html5lib         : None
pymysql          : None
psycopg2         : None
jinja2           : None
IPython          : 7.17.0
pandas_datareader: None
bs4              : None
bottleneck       : None
fastparquet      : None
gcsfs            : None
lxml.etree       : None
matplotlib       : None
numexpr          : None
odfpy            : None
openpyxl         : None
pandas_gbq       : None
pyarrow          : None
pytables         : None
s3fs             : None
scipy            : None
sqlalchemy       : None
tables           : None
xarray           : None
xlrd             : None
xlwt             : None
xlsxwriter       : None

After:

In [2]: pd.to_numeric(pd.Series([42], dtype="Int64"), downcast="integer")
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-2-0969c270a2a4> in <module>
----> 1 pd.to_numeric(pd.Series([42], dtype="Int64"), downcast="integer")

/usr/lib/python3.8/site-packages/pandas/core/tools/numeric.py in to_numeric(arg, errors, downcast)
    180             for dtype in typecodes:
    181                 if np.dtype(dtype).itemsize <= values.dtype.itemsize:
--> 182                     values = maybe_downcast_to_dtype(values, dtype)
    183 
    184                     # successful conversion

/usr/lib/python3.8/site-packages/pandas/core/dtypes/cast.py in maybe_downcast_to_dtype(result, dtype)
    149         dtype = np.dtype(dtype)
    150 
--> 151     converted = maybe_downcast_numeric(result, dtype, do_round)
    152     if converted is not result:
    153         return converted

/usr/lib/python3.8/site-packages/pandas/core/dtypes/cast.py in maybe_downcast_numeric(result, dtype, do_round)
    241                     return new_result
    242             else:
--> 243                 if np.allclose(new_result, result, rtol=0):
    244                     return new_result
    245 

<__array_function__ internals> in allclose(*args, **kwargs)

/usr/lib/python3.8/site-packages/numpy/core/numeric.py in allclose(a, b, rtol, atol, equal_nan)
   2187 
   2188     """
-> 2189     res = all(isclose(a, b, rtol=rtol, atol=atol, equal_nan=equal_nan))
   2190     return bool(res)
   2191 

<__array_function__ internals> in isclose(*args, **kwargs)

/usr/lib/python3.8/site-packages/numpy/core/numeric.py in isclose(a, b, rtol, atol, equal_nan)
   2286 
   2287     xfin = isfinite(x)
-> 2288     yfin = isfinite(y)
   2289     if all(xfin) and all(yfin):
   2290         return within_tol(x, y, atol, rtol)

TypeError: ufunc 'isfinite' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

In [3]: pd.show_versions()

INSTALLED VERSIONS
------------------
commit           : d9fff2792bf16178d4e450fe7384244e50635733
python           : 3.8.5.final.0
python-bits      : 64
OS               : Linux
OS-release       : 5.7.12-arch1-1
Version          : #1 SMP PREEMPT Fri, 31 Jul 2020 17:38:22 +0000
machine          : x86_64
processor        : 
byteorder        : little
LC_ALL           : None
LANG             : en_US.UTF-8
LOCALE           : en_US.UTF-8

pandas           : 1.1.0
numpy            : 1.19.1
pytz             : 2020.1
dateutil         : 2.8.1
pip              : 20.1.1
setuptools       : 49.6.0
Cython           : 0.29.21
pytest           : 6.0.1
hypothesis       : 5.25.0
sphinx           : 3.2.0
blosc            : None
feather          : None
xlsxwriter       : 1.3.2
lxml.etree       : 4.5.2
html5lib         : 1.1
pymysql          : None
psycopg2         : 2.8.5 (dt dec pq3 ext lo64)
jinja2           : 2.11.2
IPython          : 7.17.0
pandas_datareader: 0.10.0dev0
bs4              : 4.9.1
bottleneck       : 1.3.2
fsspec           : None
fastparquet      : None
gcsfs            : None
matplotlib       : 3.3.1
numexpr          : 2.7.1
odfpy            : None
openpyxl         : None
pandas_gbq       : None
pyarrow          : None
pytables         : None
pyxlsb           : None
s3fs             : None
scipy            : 1.5.2
sqlalchemy       : 1.3.18
tables           : None
tabulate         : None
xarray           : None
xlrd             : None
xlwt             : None
numba            : None
jorisvandenbossche

comment created time in a month

issue openedpandas-dev/pandas

DOC: downcast argument for pd.to_numeric should be integer, not int

Location of the documentation

https://pandas.pydata.org/docs/dev/reference/api/pandas.to_numeric.html?highlight=to_numeric#pandas.to_numeric

Documentation problem

The documentation declares that the downcast argument of pd.to_numeric can use the value int. However, only integer is working in the real code:

In [8]: pd.to_numeric(["42"], downcast="int", errors="coerce")
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-8-3b178e77dcd5> in <module>
----> 1 pd.to_numeric(["42"], downcast="int", errors="coerce")

/usr/lib/python3.8/site-packages/pandas/core/tools/numeric.py in to_numeric(arg, errors, downcast)
    110     """
    111     if downcast not in (None, "integer", "signed", "unsigned", "float"):
--> 112         raise ValueError("invalid downcasting method provided")
    113 
    114     if errors not in ("ignore", "raise", "coerce"):

ValueError: invalid downcasting method provided

In [9]: pd.to_numeric(["42"], downcast="integer", errors="coerce")
Out[9]: array([42], dtype=int8)

Suggested fix for documentation

Replace int with integer

created time in a month

delete branch languitar/pass-git-helper

delete branch : feature/improve-documentation-for-pinentry-curses

delete time in a month

push eventlanguitar/pass-git-helper

Johannes Wienke

commit sha ae71bf684f27b69af8c9f473f0088cc1f3df6f0f

Improve the documentation for curses pinentry Will fix #18.

view details

Johannes Wienke

commit sha 3fe67e004898bfeba5edea9d83777f3d82e94a9b

Merge pull request #19 from languitar/feature/improve-documentation-for-pinentry-curses Improve the documentation for curses pinentry

view details

push time in a month

push eventlanguitar/xrays

Johannes Wienke

commit sha df3585ee8cbbdff0e597bf7d4382fa5b0d555227

Use tabs

view details

push time in a month

push eventlanguitar/xrays

Johannes Wienke

commit sha cc08757d7dc09cc45b7107b23d1ce6c6f1b60653

Split into separate modules

view details

push time in a month

pull request commentlanguitar/pass-git-helper

Improve the documentation for curses pinentry

@lervag does this help for resolving #18 ?

languitar

comment created time in a month

issue closedlanguitar/pass-git-helper

Passing username via configuration directly?

Looking at the README and code, am I correct in that there isn't currently a way to pass username directly from the git-pass-mappinig.ini?

For example,

[github.com*]
target=software/development/github
username=tfurf

If that understanding is correct, would a PR be welcome to add such functionality?

closed time in a month

tfurf

issue commentlanguitar/pass-git-helper

Error when using through ssh

Btw, from the gpg agent documentation

You should always add the following lines to your .bashrc or whatever initialization file is used for all shell invocations:

GPG_TTY=$(tty)
export GPG_TTY
lervag

comment created time in a month

issue commentlanguitar/pass-git-helper

Error when using through ssh

I think the original problem is not my redirection, but the one that git does when communicating with passgithelper via stdin and stdout.

lervag

comment created time in a month

issue commentlanguitar/pass-git-helper

Error when using through ssh

I think I know why it works in my case. Please try this before running the git operation:

export GPG_TTY=$(tty)
lervag

comment created time in a month

issue commentlanguitar/pass-git-helper

Error when using through ssh

Note, though: When I do pass show ..., I do get the curses interface for entering the gpg pass key.

This is somehow related to the output redirection that is necessary to get the output from pass show in the script. But I don't see, why this works on my machine and not on yours.

lervag

comment created time in a month

issue commentlanguitar/pass-git-helper

Error when using through ssh

so weird. Which distro and version is running on the server?

Btw, maybe GPG agent forwarding is running on the remote host?

lervag

comment created time in a month

issue commentlanguitar/pass-git-helper

Error when using through ssh

Alright, let's try to replicate my setup on you side, just to be sure. Please try the following git config:

[credential]
    helper = !/home/languitar/src/pass-git-helper/passgithelper.py -l $@

With an appropriate path. -l will add some debug logging but has the potential to log confidential information. Please use some test account if possible.

Moreover, please use the following gpg-agent.conf on the server side:

pinentry-program /usr/bin/pinentry-curses

Before testing anything, please use:

killall gpg-agent dirmngr gpg

to be sure that a fresh gpg-agent is used.

What does happen now, if you do some git clone https://...?

lervag

comment created time in a month

issue commentlanguitar/pass-git-helper

Error when using through ssh

Sorry, this took a lot longer than I wanted to... One more question: do you use plan SSH keys or do you use GPG for SSH authentication?

lervag

comment created time in a month

issue commentmongodb/terraform-provider-mongodbatlas

mongodbatlas_maintenance_window fails with BadRequest: Invalid Day of Week

Good to see that this could be replicated. Did you also get the Atlas Error for the wrong parameter for the day of the week? I can't see that in your diff.

languitar

comment created time in a month

issue commentmongodb/terraform-provider-mongodbatlas

Renaming a custom_db_role with attached users is not possible

Here is is: https://feedback.mongodb.com/forums/924145-atlas/suggestions/41140231-indicate-operation-errors-already-in-the-plan-phas

I have no idea how to properly format the code snippet there.

languitar

comment created time in a month

issue commentmongodb/terraform-provider-mongodbatlas

mongodbatlas_maintenance_window fails with BadRequest: Invalid Day of Week

We had an existing configuration in Atlas with exactly the settings as described in the HCL file. A terraform configuration didn't exist yet. Therefore, I imported the resource and afterwards issued a plan and apply cycle.

Also note, while the maintenance window is set at the project level it won't impact shared tier clusters, only M10 or above.

Good to know. But not a real issue so far. Using the terraform-based configuration here is mostly just a test for applying this also for the production cluster.

languitar

comment created time in a month

issue commentmongodb/terraform-provider-mongodbatlas

Renaming a custom_db_role with attached users is not possible

I understand this. But the feature request template was included in the repo right after posted this issue, so I couldn't use that one ;-)

languitar

comment created time in a month

issue commentmongomock/mongomock

create_index doesn't raise an error in case of duplicate indexes

Thanks for fixing this!

languitar

comment created time in a month

issue commentmongomock/mongomock

Invalid arguments to create_index are not captured

I'm fine with that here. This is more a rant about the pymongo implementation details leaking through their API ;-)

languitar

comment created time in 2 months

startedadamchainz/flake8-comprehensions

started time in 2 months

issue commentmongomock/mongomock

Invalid arguments to create_index are not captured

Thanks for quickly handling this. Btw. the TypeError used by pymongo also isn't very informative:

TypeError: not all arguments converted during string formatting

languitar

comment created time in 2 months

issue openedmongodb/terraform-provider-mongodbatlas

mongodbatlas_maintenance_window fails with BadRequest: Invalid Day of Week

Terraform CLI and Terraform MongoDB Atlas Provider Version

Terraform v0.12.27
+ provider.mongodbatlas v0.6.3

Terraform Configuration File

resource "mongodbatlas_maintenance_window" "maintenance" {
  project_id  = data.mongodbatlas_project.this.id
  day_of_week = 2
  hour_of_day = 4
}

Steps to Reproduce

I have remodeled the existing configuration of one of our Atlas clusters (shared tier, M5). One aspect of this is the maintenance window. For that, I am using the configuration shown above. Interestingly, the initial plan resulted in this diff (which I don't understand):

  # mongodbatlas_maintenance_window.maintenance will be updated in-place
  ~ resource "mongodbatlas_maintenance_window" "maintenance" {
        day_of_week         = 2
        hour_of_day         = 4
        id                  = "<theid>"
        number_of_deferrals = 0
      + project_id          = "<theid>"
        start_asap          = false
    }

Expected Behavior

No diff is shown because the actual settings match.

Actual Behavior

Applying the above plan results in the following error:

Error: error updating the MongoDB Atlas Maintenance Window (<someid>): PATCH https://cloud.mongodb.com/api/atlas/v1.0/groups/<theid>/maintenanceWindow: 400 (request "Bad Request") Invalid Day of Week. Value should be between 1 and 7 (Sunday = 1).

Debug Output

Unfortunately, I can't reproduce this now, as a new iteration of planing doesn't yield a diff anymore.

created time in 2 months

issue openedmongomock/mongomock

create_index doesn't raise an error in case of duplicate indexes

MongoDB itself cannot create multiple indexes with the same set of fields and sorting options. However, create_index in mongomock doesn't replicate this behavior:

In [17]: import pymongo                                                                                                                                                                                                                                                                     

In [18]: real_client = pymongo.MongoClient("mongodb://root:rootpass@localhost/")                                                                                                                                                                                                            

In [19]: real_collection = real_client.test.index_test                                                                                                                                                                                                                                      

In [20]: real_collection.create_index([("foo", -1)])                                                                                                                                                                                                                                        
Out[20]: 'foo_-1'

In [21]: real_collection.create_index([("foo", -1)], expireAfterSeconds=42)                                                                                                                                                                                                                 
---------------------------------------------------------------------------
OperationFailure                          Traceback (most recent call last)
<ipython-input-21-7f046f171dde> in <module>
----> 1 real_collection.create_index([("foo", -1)], expireAfterSeconds=42)

~/src/preprod-sync/.venv/lib/python3.8/site-packages/pymongo/collection.py in create_index(self, keys, session, **kwargs)
   1993         if "maxTimeMS" in kwargs:
   1994             cmd_options["maxTimeMS"] = kwargs.pop("maxTimeMS")
-> 1995         self.__create_index(keys, kwargs, session, **cmd_options)
   1996         return name
   1997 

~/src/preprod-sync/.venv/lib/python3.8/site-packages/pymongo/collection.py in __create_index(self, keys, index_options, session, **kwargs)
   1888             cmd = SON([('createIndexes', self.name), ('indexes', [index])])
   1889             cmd.update(kwargs)
-> 1890             self._command(
   1891                 sock_info, cmd, read_preference=ReadPreference.PRIMARY,
   1892                 codec_options=_UNICODE_REPLACE_CODEC_OPTIONS,

~/src/preprod-sync/.venv/lib/python3.8/site-packages/pymongo/collection.py in _command(self, sock_info, command, slave_ok, read_preference, codec_options, check, allowable_errors, read_concern, write_concern, collation, session, retryable_write, user_fields)
    233         """
    234         with self.__database.client._tmp_session(session) as s:
--> 235             return sock_info.command(
    236                 self.__database.name,
    237                 command,

~/src/preprod-sync/.venv/lib/python3.8/site-packages/pymongo/pool.py in command(self, dbname, spec, slave_ok, read_preference, codec_options, check, allowable_errors, check_keys, read_concern, write_concern, parse_write_concern_error, collation, session, client, retryable_write, publish_events, user_fields)
    601             self._raise_if_not_writable(unacknowledged)
    602         try:
--> 603             return command(self.sock, dbname, spec, slave_ok,
    604                            self.is_mongos, read_preference, codec_options,
    605                            session, client, check, allowable_errors,

~/src/preprod-sync/.venv/lib/python3.8/site-packages/pymongo/network.py in command(sock, dbname, spec, slave_ok, is_mongos, read_preference, codec_options, session, client, check, allowable_errors, address, check_keys, listeners, max_bson_size, read_concern, parse_write_concern_error, collation, compression_ctx, use_op_msg, unacknowledged, user_fields)
    163                 client._process_response(response_doc, session)
    164             if check:
--> 165                 helpers._check_command_response(
    166                     response_doc, None, allowable_errors,
    167                     parse_write_concern_error=parse_write_concern_error)

~/src/preprod-sync/.venv/lib/python3.8/site-packages/pymongo/helpers.py in _check_command_response(response, msg, allowable_errors, parse_write_concern_error)
    157 
    158             msg = msg or "%s"
--> 159             raise OperationFailure(msg % errmsg, code, response)
    160 
    161 

OperationFailure: Index with name: foo_-1 already exists with different options

Mongomock simply allows both calls to pass.

created time in 2 months

issue openedmongomock/mongomock

Invalid arguments to create_index are not captures

Currently, you can construct illegal indexes that later break for instance list_indexes:

In [1]: import mongomock                                                                                                                                                                                                                                                                    

In [2]: client = mongomock.MongoClient()                                                                                                                                                                                                                                                    

In [3]: collection = client.foo.bar                                                                                                                                                                                                                                                         

In [4]: collection.create_index([("one", -1, "foo", "bar")])                                                                                                                                                                                                                                
Out[4]: 'one_-1_foo_bar'

In [5]: next(collection.list_indexes())                                                                                                                                                                                                                                                     
Out[5]: {'key': {'_id': 1}, 'name': '_id_', 'v': 2, 'ns': 'foo.bar'}

In [6]: list(collection.list_indexes())                                                                                                                                                                                                                                                     
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-6-59c676dc5fe4> in <module>
----> 1 list(collection.list_indexes())

~/src/preprod-sync/.venv/lib/python3.8/site-packages/mongomock/collection.py in list_indexes(self, session)
   1473             yield dict(
   1474                 information,
-> 1475                 key=dict(information['key']),
   1476                 name=name,
   1477                 v=2,

ValueError: dictionary update sequence element #0 has length 4; 2 is required

created time in 2 months

issue commentmrparkers/terraform-provider-keycloak

allow for mapping roles to users and service accounts

As the #247 is closed now and #315 is merged, is this resolved now?

mrparkers

comment created time in 2 months

issue openedmongodb/terraform-provider-mongodbatlas

Renaming a custom_db_role with attached users is not possible

Changing the name of an existing custom_db_role is currently not possible without ending in an error in the apply step. The plan for a name change currently indicates the replacement due to the name change:

  # module.versioner.mongodbatlas_custom_db_role.this must be replaced
-/+ resource "mongodbatlas_custom_db_role" "this" {
      ~ id         = "someid" -> (known after apply)
        project_id = "5c860ed2a6f2396cd47f4785"
      ~ role_name  = "old_name" -> "newName" # forces replacement

Applying this results in the following error:

Error: error deleting custom db role (mongoversioner): DELETE https://cloud.mongodb.com/api/atlas/v1.0/groups/projectid/customDBRoles/roles/old_name: 409 (request "Conflict") Deleting specified custom role would leave the following users without a role: someuser.

I understand that this is not possible directly without either unassigning the user, but some indication for this issue already in the plan phase would make things much easier to handle.

created time in 2 months

issue commentmongodb/terraform-provider-mongodbatlas

inherited_roles are not correctly removed from custom_db_roles

Alright, as my comment on #279 is probably more related to this issue, here it is again:

Another aspect of this thing could be the following:

      ~ inherited_roles {
          ~ database_name = "history" -> "userdata"
            role_name     = "readWrite"
        }
        inherited_roles {
            database_name = "catalog"
            role_name     = "readWrite"
        }
      ~ inherited_roles {
          ~ database_name = "userdata" -> "cache"
            role_name     = "readWrite"
        }
      ~ inherited_roles {
          ~ database_name = "cache" -> "history"
            role_name     = "readWrite"
        }

Seems the sorting wasn't correctly applied in the previous iteration and now this diff reappears everytime.

languitar

comment created time in 2 months

issue commentmongodb/terraform-provider-mongodbatlas

custom_db_roles cannot be created with only inherited roles

Another aspect of this thing could be the following:

      ~ inherited_roles {
          ~ database_name = "history" -> "userdata"
            role_name     = "readWrite"
        }
        inherited_roles {
            database_name = "catalog"
            role_name     = "readWrite"
        }
      ~ inherited_roles {
          ~ database_name = "userdata" -> "cache"
            role_name     = "readWrite"
        }
      ~ inherited_roles {
          ~ database_name = "cache" -> "history"
            role_name     = "readWrite"
        }

Seems the sorting wasn't correctly applied in the previous iteration and now this diff reappears everytime.

languitar

comment created time in 2 months

CommitCommentEvent
more