profile
viewpoint

Ask questionsBoto3 causing segmentation fault

In some special circumstance, using boto3 will cause segmentation fault.

The code that can reproduce it is as follows. There are totally 3 files (and one subdirectory) need to be created because I am not able to reproduce it with only one file. You can also clone from https://github.com/JasonXJ/segmentation_fault_with_boto3

./mypackage/__init__.py:

import sys
import boto3

def foo():
    bar(boto3.resource('dynamodb', region_name='us-east-2'))
    sys.exit(0)  # Unable to reproduce without this line


# Unable to reproduce if merge this function with foo()
def bar(dynamodb):
    import mypackage.justtyping  # Unable to reproduce without this line

    # Unable to reproduce if do not get item twice, or use `dynamodb` in both
    # cases, or create new resource in both cases.
    # Note that the table must exist, but it does not need to have the item.
    dynamodb.Table('sigsegv').get_item(Key={'pkey': 'xxx'})
    boto3.resource('dynamodb', region_name='us-east-2').Table('sigsegv').get_item(Key={'pkey': 'xxx'})

./mypackage/justtyping.py

from typing import Sequence
x = Sequence['xxx']  # Unable to reproduce without this line, or replace string `'xxx'` with a class (e.g. `bytes`)

./run.py

import mypackage
mypackage.foo()

Run the code and get the segmentation fault:

$ python3 run.py
[1]    26495 segmentation fault (core dumped)  python3 run.py
FAIL

Versions: python 3.5.2, boto3 1.4.4, ubuntu 16.04.2

boto/boto3

Answer questions chrisdlangton

i am also seeing segfault #2534 but only when using S3, in my testing i went back to python 3.7 and saw the error existed so i ruled out it being a python bug and didn't go back further.

I think there might be a commonality somewhere in boto3 session code paths, if we assume there is a connection.

useful!

Related questions

AttributeError: module 'boto3' has no attribute 'resource' hot 3
Waiter encountered a terminal failure state hot 2
Add resource waiter for StackSets hot 1
DatabaseMigrationService.Waiter.ReplicationTaskStopped doesn't work for recently started jobs hot 1
ImportError: cannot import name ReadTimeoutError when trying to create s3 client hot 1
Excessive memory usage on multithreading hot 1
Pre-signed URL issues (generate_presigned_post and generate_presigned_url) hot 1
Boto3 incompatible with python zip import hot 1
Boto does not recognize ~.aws/credentials hot 1
New region ap-east-1 causes issues in describe_instances hot 1
InvalidParameterException on ECS update service hot 1
"Unknown service: 'rds-data'. hot 1
Problems with ec2.describe_fleet_history hot 1
AWS Batch and AWS Transcribe: LimitExceededException botocore.errorfactory.LimitExceededException - boto3 hot 1
Exceptions can't be caught with try/except hot 1
source:https://uonfu.com/
Github User Rank List