profile
viewpoint

Considerit/ConsiderIt 27

For deliberation and opinion visualization

alexpatow/xlsform-api 0

XLSForm API is a Flask web service that internally uses xls2xform for converting Excel.

yanokwa/aws-lambda-ses-forwarder 0

Serverless email forwarding using AWS Lambda and SES

yanokwa/build2xlsform 0

service to convert odkbuild-form json to xlsform-compliant xlsx.

yanokwa/BulkScanDownloader 0

A Java command-line version of the Earth Class Mail Bulk Scan Downloader program.

Pull request review commentgetodk/collect

Allow user to recover from multiple forms with same formid/version and prevent their future download

 private void loadFromIntent(Intent intent) {             }              instancePath = formInfo.getInstancePath();+            List<Form> candidateForms = formsRepository.getAll(formInfo.getFormId(), formInfo.getFormVersion()); -            String jrFormId = formInfo.getFormId();-            String jrVersion = formInfo.getFormVersion();--            String[] selectionArgs;-            String selection;-            if (jrVersion == null) {-                selectionArgs = new String[]{jrFormId};-                selection = FormsColumns.JR_FORM_ID + "=? AND "-                        + FormsColumns.JR_VERSION + " IS NULL";-            } else {-                selectionArgs = new String[]{jrFormId, jrVersion};-                selection = FormsColumns.JR_FORM_ID + "=? AND "-                        + FormsColumns.JR_VERSION + "=?";-            }--            int formCount = FormsDaoHelper.getFormsCount(selection, selectionArgs);-            if (formCount < 1) {+            if (candidateForms.isEmpty()) {                 createErrorDialog(getString(                         R.string.parent_form_not_present,-                        jrFormId)-                                + ((jrVersion == null) ? ""-                                : "\n"-                                + getString(R.string.version)-                                + " "-                                + jrVersion),+                        formInfo.getFormId())+                                + ((formInfo.getFormVersion() == null) ? ""+                                : "\n" + getString(R.string.version) + " " + formInfo.getFormVersion()),                         EXIT);                 return;-            } else {-                formPath = FormsDaoHelper.getFormPath(selection, selectionArgs);--                /**-                 * Still take the first entry, but warn that there are multiple rows. User will

This just wasn't true. The dialog errored out and quit the form so this wasn't a warning and the first entry was never used. Also it's no longer true that the SQLite database needs to be touched directly.

lognaturel

comment created time in an hour

Pull request review commentgetodk/collect

Allow user to recover from multiple forms with same formid/version and prevent their future download

 public static EncryptedFormInformation getEncryptedFormInformation(Uri uri,         Cursor formCursor = null;         try {             if (InstanceColumns.CONTENT_ITEM_TYPE.equals(cr.getType(uri))) {-                // chain back to the Form record...-                String[] selectionArgs = null;-                String selection = null;-                Cursor instanceCursor = null;-                try {-                    instanceCursor = cr.query(uri, null, null, null, null);+                String[] selectionArgs;+                String selection = FormsColumns.DELETED_DATE + " IS NULL AND " + FormsColumns.JR_FORM_ID + " =? AND ";+                try (Cursor instanceCursor = cr.query(uri, null, null, null, null)) {

I really wanted to get this using the instance/form repositories but it didn't feel worth the effort and risk at this time.

lognaturel

comment created time in an hour

Pull request review commentgetodk/collect

Allow user to recover from multiple forms with same formid/version and prevent their future download

     private final String formsDirPath;     private final FormMetadataParser formMetadataParser; -    public ServerFormDownloader(FormSource formSource, FormsRepository formsRepository, File cacheDir, String formsDirPath, FormMetadataParser formMetadataParser) {+    private final Analytics analytics;++    public ServerFormDownloader(FormSource formSource, FormsRepository formsRepository, File cacheDir, String formsDirPath, FormMetadataParser formMetadataParser, Analytics analytics) {         this.cacheDir = cacheDir;         this.formsDirPath = formsDirPath;         this.multiFormDownloader = new MultiFormDownloader(formsRepository, formSource);         this.formsRepository = formsRepository;         this.formMetadataParser = formMetadataParser;++        this.analytics = analytics;     }      @Override     public void downloadForm(ServerFormDetails form, @Nullable ProgressReporter progressReporter, @Nullable Supplier<Boolean> isCancelled) throws FormDownloadException, InterruptedException {         Form formOnDevice = formsRepository.get(form.getFormId(), form.getFormVersion());-        if (formOnDevice != null && formOnDevice.isDeleted()) {-            formsRepository.restore(formOnDevice.getId());+        if (formOnDevice != null) {+            if (formOnDevice.isDeleted()) {+                formsRepository.restore(formOnDevice.getId());+            } else if (!(form.getHash().equals(formOnDevice.getMD5Hash()))) {+                String formIdentifier = formOnDevice.getDisplayName() + " " + formOnDevice.getId();+                String formIdHash = FileUtils.getMd5Hash(new ByteArrayInputStream(formIdentifier.getBytes()));+                analytics.logFormEvent(DOWNLOAD_SAME_FORMID_VERSION, formIdHash);+                throw new FormDownloadException("You've already downloaded a form with the same ID and version but with different contents. " +

I went back and forth on whether to localize the download message. I think it's uncommon enough that we don't need to. I've added analytics so that we can verify that claim. I also hope that if it is more common than I expect we'll hear from users on the forum and perhaps we'll choose to work on the wording. "please send all data you have collected with the existing form and delete the data and blank form" is admittedly awkward and I'm open to alternatives.

lognaturel

comment created time in an hour

Pull request review commentgetodk/collect

Allow user to recover from multiple forms with same formid/version and prevent their future download

 public Form get(Long id) {     @Nullable     @Override     public Form get(String jrFormId, @Nullable String jrVersion) {-        if (jrVersion != null) {-            return queryForForm(JR_FORM_ID + "=? AND " + JR_VERSION + "=?", new String[]{jrFormId, jrVersion});+        List<Form> all = getAll(jrFormId, jrVersion);+        if (!all.isEmpty()) {+            return all.get(0);

This is a change in behavior because now deleted forms will be excluded. That seems like the desired behavior to me. Also note that if there is a formid/version conflict, we'll just get the first one. That hasn't changed.

lognaturel

comment created time in an hour

Pull request review commentgetodk/collect

Allow user to recover from multiple forms with same formid/version and prevent their future download

 public DatabaseFormsRepository() {         }     } +    @Override+    public List<Form> getAll(String jrFormId, @Nullable String jrVersion) {

I'm not sure about naming, here. There's getByJrFormIdNotDeleted above. But I can't think of a case where we'd want deleted forms so calling it getAll seemed right to me.

lognaturel

comment created time in an hour

PR opened getodk/collect

Allow user to recover from multiple forms with same formid/version and prevent their future download

Closes #4237

Strongly recommend reviewing commit-by-commit.

If this were not a legacy project we'd want to use a hash to represent a unique form. An improvement we could make is linking instances and the forms they were filled from through database ids rather than through formid/version. However, that's an invasive change that I don't think we should take on so close to a planned release. It's probably not reasonable to take on at all before some of the mission-critical parts of the code have better test coverage.

What has been done to verify that this works as intended?

Added automated tests for the changes to form downloads and delete behavior. I don't think there are low level tests to write for the changes to instance load and finalizing given the current code structure and I didn't want to get into a risky refactor. I think the problem is enough of an edge-case that I don't want to introduce Espresso tests. I verified those manually for now.

Cases

  • adb push form with same formid/version and same filename. Collect should replace no matter the contents. Can get in a bad state if you have filled forms for one older version and there are schema changes. Won't be able to open the filled forms.
  • adb push two identical forms (same md5 hash) but different filenames. Collect will treat these as different and this will cause the bad state from the issue.
  • adb push same formid/version but the form contents are different (different md5 hashes). Collect will treat these as different and this will cause the bad state from the issue.
  • try to download form with same formid/version and same filename. Collect should download and replace. There may or may not be new media attachments.
  • try to download form with same formid/version/contents but different filenames. Collect should download but treat as different form. This will cause the bad state from the issue.
  • try to download form with same formid/version/filename but different contents. Collect should prevent download.
  • get a form on the device. Fill it out a few times. Delete it. Get a form with the same
  • got into a bad state in v1.28 through one of the ways above, soft deleted one of the forms but was still suck
  • got into a bad state in v1.28 through one of the ways above, had not yet deleted one of the forms

Why is this the best possible solution? Were any other approaches considered?

I think this is a good balance of not too invasive and pretty thorough. It both allows folks who ended up in a bad state in v1.28 to recover and limits the ways that the bad state can be reached in the future.

The first two commits filter out soft deleted forms when finalizing or opening filled forms. After those, it should be possible to recover from the bad state by deleting one of the blank forms. This helps folks who already ended up with the problem in v1.28 and soft-deleted one of the copies as instructed.

The third commit prevents downloading a form with the same formid/version as a form already on the device but with different contents. It still allows downloading a form with the same formid/version and the same contents because some servers will allow that to be combined with a media file update (Aggregate but not Central). This should make it harder for users to get into the bad state. There is still the possibility that the remote form would have the same contents but a different filename. We can't guard against that before downloading because the form XML endpoint may not include the filename. We'd have to do the check after downloading the file and I don't think this is common enough to add a special case. Plus, the two first commits will allow users to recover.

I went back and forth on whether to localize the download message. I think it's uncommon enough that we don't need to. I'd also like to see whether it's confusing to users before we do.

How does this change affect users? Describe intentional changes to behavior and behavior that could have accidentally been affected by code changes. In other words, what are the regression risks?

This affects form load, form finalization, form download and form delete. There are opportunities for regression in any of these but I tried to make the changes easy to review to mitigate that. The challenge is that there are a lot of possible combinations to think of. I tried to list those above in the verification section.

Do we need any specific form for testing your changes? If so, please attach one.

No, any form will do.

Does this change require updates to documentation? If so, please file an issue here and include the link below.

I don't think so. It doesn't seem common enough to mention.

Before submitting this PR, please make sure you have:

  • [ ] run ./gradlew checkAll and confirmed all checks still pass OR confirm CircleCI build passes and run ./gradlew connectedDebugAndroidTest locally.
  • [x] verified that any code or assets from external sources are properly credited in comments and/or in the about file.
  • [x] verified that any new UI elements use theme colors. UI Components Style guidelines
+172 -65

0 comment

16 changed files

pr created time in an hour

issue commentgetodk/collect

Help/guidance/warning area should be hidden when there is no content

#4130 addressed it. I think it was left open because of:

It'd also be good to revise the spacing once this change is made to make sure there is still enough space between the label/help and the answer in all the widgets. We should add spacing to widgets individually if they feel cramped.

I have not noticed any issues in my use. I think it would be good for @getodk/testers to keep an eye out for any spacing improvements they might suggest as they do the regression pass.

seadowg

comment created time in 2 hours

issue closedgetodk/collect

Help/guidance/warning area should be hidden when there is no content

Software and hardware versions

Collect v1.28.1 (and earlier)

Problem description

The view that shows help, guidance and warning text still takes up space when there is no content displayed in it.

Steps to reproduce the problem

  1. Make a form with a question that has no help, guidance or warning
  2. Scroll to the question

You'll see there is some extra space between the question label and the answer.

Expected behavior

Extra space (view) should disappear when there is nothing to display in it.

Other information

It'd also be good to revise the spacing once this change is made to make sure there is still enough space between the label/help and the answer in all the widgets. We should add spacing to widgets individually if they feel cramped.

closed time in 2 hours

seadowg

issue commentgetodk/collect

Help/guidance/warning area should be hidden when there is no content

The issue does not exist according to my investigation:

Hint and guidance No hint and no guidance
2 1

layoutTest.xlsx

seadowg

comment created time in 2 hours

issue commentgetodk/collect

Handle errors while copying recording

Here's another interesting possible error for built-in audio: certain devices might not have the codecs we think they should. Seems like manufacturers might take some out, possibly to avoid royalties. I saw someone saying this about certain low-end manufacturers when looking for something else but am not finding the reference again. https://stackoverflow.com/a/12369710/137744 makes the claim as well.

seadowg

comment created time in 2 hours

pull request commentgetodk/collect

Fixed: User is able to open two different views from the main screen

Is the implementation problematic or just the usage? Does this affect anything else?

getClass().getName() used in if (MultiClickGuard.allowClick(getClass().getName())) was returning numbers at the end of class names like 1,2,3 (because it's inside anonymous classes) for different buttons that's why it didn't work.

grzesiek2010

comment created time in 2 hours

Pull request review commentgetodk/collect

Add pause/unpause to recording

 internal open class AudioRecorderDependencyModule {      @Provides     open fun providesRecorder(application: Application): Recorder {-        return MediaRecorderRecorder(application.cacheDir) { RealMediaRecorderWrapper(MediaRecorder()) }+        return RecordingResourceRecorder(application.cacheDir) { MediaRecorderRecordingResource(MediaRecorder()) }

Naming is hard. I'm attempting to make it clear that this could be implemented using something other than MediaRecorder. It's the direction I think we'd need to take if we look at a lossless codec.

seadowg

comment created time in 3 hours

pull request commentgetodk/collect

Fixed: Minimal appearance doesn't work with "fast external itemsets"

Tested with success

Verified on Android versions: 7.0, 8.1, 10.0

Verified cases:

  • selectOneExternal form - cascading select with minimal appearance
  • cascading select external itemsets form
  • standard cascading select
  • move forward/backward
  • jump from the hierarchy view
  • removed response - new issue created
  • change response - new issue created
  • use search
  • form edit
  • light/dark mode
  • RTL language
  • rotate screen
  • minimize app
grzesiek2010

comment created time in 3 hours

push eventgetodk/collect

Callum Stott

commit sha b652335ebffea0783431e4aaa61f742dae06a5f5

Make external recording the default

view details

Hélène Martin

commit sha bbb2c28b0e34d11af18090c5fce80d11f610fc53

Merge pull request #4255 from seadowg/external-default Make external recording the default

view details

push time in 3 hours

PR merged getodk/collect

Make external recording the default

Closes #4254

What has been done to verify that this works as intended?

Changed tests and verified manually

Why is this the best possible solution? Were any other approaches considered?

Just reversed the default so no choices here really.

How does this change affect users? Describe intentional changes to behavior and behavior that could have accidentally been affected by code changes. In other words, what are the regression risks?

Should be able to skip QA for this as it's easy to verify when we're playing with the app.

Before submitting this PR, please make sure you have:

  • [x] run ./gradlew checkAll and confirmed all checks still pass OR confirm CircleCI build passes and run ./gradlew connectedDebugAndroidTest locally.
  • [x] verified that any code or assets from external sources are properly credited in comments and/or in the about file.
  • [x] verified that any new UI elements use theme colors. UI Components Style guidelines
+12 -12

1 comment

4 changed files

seadowg

pr closed time in 3 hours

issue closedgetodk/collect

External recording is the default

Acceptance

  • [ ] Given I install Collect When I start recording with an audio question Then it uses an external app to record

  • [ ] Given I install Collect When I go to Form Entry settings Then "Use external app for recording" is enabled

closed time in 3 hours

seadowg

pull request commentgetodk/collect

Make external recording the default

@getodk/testers The default audio mode will be external for this release so we don't affect users' workflows and we have an opportunity for more feedback before making internal recording the default. No need to verify this PR but you'll see that change when you see any audio questions. A form-specified quality like voice-only or normal still forces internal recording.

seadowg

comment created time in 3 hours

pull request commentgetodk/collect

Fixed: User is able to open two different views from the main screen

using MultiClickGuard.allowClick in OnClickListener was faulty.

Is the implementation problematic or just the usage? Does this affect anything else?

grzesiek2010

comment created time in 3 hours

pull request commentgetodk/collect

Show recording duration and waveform

@getodk/testers Heads up that we'll get these features to you after a few more visual tweaks!

seadowg

comment created time in 4 hours

push eventgetodk/collect

Callum Stott

commit sha dd1248051c363bcf9b3e7f7b9ed12fee205edefd

Show recording duration in notification

view details

Callum Stott

commit sha 433476731ff8c8580c063c212ce866ac85061caa

Move notification code out of service

view details

Callum Stott

commit sha 0f530ac4dab9416c8f1bb47f653149f718c29c85

Hook up duration but no updates yet

view details

Callum Stott

commit sha abdd668c17db4d604935f98ae0afd42a69f7be40

Update duration every second

view details

Callum Stott

commit sha be8204151c78fda5195b784fa8272f08e696f0b3

Move all foreground service logic into notification class

view details

Callum Stott

commit sha 198c07129bb951daa1613a07d4c13c182001b141

Visual tweaks based on mockups

view details

Callum Stott

commit sha 81d75ebcb5406bf6078929526000561cd5cd22b6

Move LengthFormatter to strings module

view details

Callum Stott

commit sha 8cbc842d09927b71b201042a6fa438c7a8e9eaf6

Add strings to Circle config

view details

Callum Stott

commit sha 248a12f42b9726f690eeec0b78fff269dccc1ec7

Simplify audio recorder interface

view details

Callum Stott

commit sha 788a0b8a51395b9ff2972f38f3227a04687854c4

Fix test missing scrolling

view details

Callum Stott

commit sha ef1f27d631288644fe027d2dcccde728a97de763

Make sure notification doesn't show after recoring stops

view details

Callum Stott

commit sha c70687c21c17e687090e60173fd86faeede4dae6

Generalize in progress recording API for widget

view details

Callum Stott

commit sha d1506a467342ad971741585b1086e7f21e4c76a2

Add waveform to AudioWidget

view details

Callum Stott

commit sha 007b6e1b96971c95c9fbd3b8769e17fb2a108ffe

Wrap AudioRecordView to make it testable and protect AudioWidget from it

view details

Callum Stott

commit sha 71fc627b45b57b370e1f186bb1bef07d4d22d5c0

Remove unused dependency

view details

Callum Stott

commit sha 74dca6c456a25bdf4c23667cbd8602eddf9392df

Tweak waveform

view details

Callum Stott

commit sha 90a78f5277f536e526ee69ae8089123e6911948f

Fix rotation for waveform

view details

Hélène Martin

commit sha fa3642314b1cfcebed4cec1a1d66458026629a1f

Merge pull request #4241 from seadowg/timecode

view details

push time in 4 hours

PR merged getodk/collect

Show recording duration and waveform

Closes #4152 Closes #4153 Blocked by #4222

What has been done to verify that this works as intended?

New tests and verified manually.

Why is this the best possible solution? Were any other approaches considered?

Comments inline.

How does this change affect users? Describe intentional changes to behavior and behavior that could have accidentally been affected by code changes. In other words, what are the regression risks?

I think we want to skip QA on this as it really only touches the new features.

Before submitting this PR, please make sure you have:

  • [x] run ./gradlew checkAll and confirmed all checks still pass OR confirm CircleCI build passes and run ./gradlew connectedDebugAndroidTest locally.
  • [x] verified that any code or assets from external sources are properly credited in comments and/or in the about file.
  • [x] verified that any new UI elements use theme colors. UI Components Style guidelines
+905 -412

1 comment

48 changed files

seadowg

pr closed time in 4 hours

issue closedgetodk/collect

Can see a waveform while recording

Acceptance

  • [x] Given I'm recording using the in-app recorder Then I see a waveform that reflects the noise around me

closed time in 4 hours

seadowg

issue closedgetodk/collect

Can see how long a recording has been going on

Acceptance

  • [x] Given I'm recording using the in-app recorder Then I see a timestamp of how long my recording has been going on And it updates every second

  • [x] Given I'm recording using the in-app recorder When I pull down the notification drawer on my device Then I see a timestamp of how long my recording has been going on in the "Recording in progress" notification And it updates every second

closed time in 4 hours

seadowg

issue openedgetodk/collect

Read-only Ex printer widget on the field-list has no spacing below hint

Software and hardware versions

Collect 1.27.3 (older versions were not verified)

Problem description

Read-only Ex printer widget on the field-list has no spacing below hint

Steps to reproduce the problem

  1. User starts a form with read-only Ex printer widget on the field-list 4228all-widgets-read-only-field-list.xml.txt
  2. User sees Ex printer widget
  3. Question has no spacing below hint

Screenshot_20201202-165004

Expected behavior

Consistent appearance for all widgets is expected. Space below the question is visible.

created time in 5 hours

issue openedgetodk/collect

The keyboard is opened automatically for read-only single/multiple select widgets when the search field is not available

Software and hardware versions

Collect with #4228

Problem description

The keyboard is opened automatically for read-only single/multiple select widgets when the search field is not available

Steps to reproduce the problem

  1. User starts form 4228all-widgets-read-only.xml.txt
  2. User goes to read-only Select one widget or Select multi widget with autocomplete appearance
  3. The search field is not available but the keyboard is opened automatically
master pr/4228
Screenshot_20201202-163005 Screenshot_20201202-163000

Expected behavior

The keyboard is not opened when the search field is not available

created time in 6 hours

pull request commentgetodk/collect

Fixed: Read-only binary questions have no spacing below the question text

Tested with success

Verified on Android versions: 7.0, 8.1, 10.0

Verified cases:

  • read only widgets - the difference is visible especially in widgets:
    • Range
    • Video
    • Data/time
    • Select one/multi. Keyboard on autocomplete appearance reported as a separate issue.
    • Trigger
  • read only widgets on field-list - Additional issue noticed also on master branch - no space below ex printer widget.
  • read only widgets on field-list with answers
  • light/dark mode
  • RTL language
  • go forward/backward
  • different font size settings
  • rotate screen
  • jump from hierarchy
  • minimize app
grzesiek2010

comment created time in 6 hours

issue openedgetodk/collect

Answers in related lower-level questions in "fast external itemsets" with the minimal appearance are not cleared when the upper-level answer is removed or changed

Software and hardware versions

Collect with #4242

Problem description

Answers in related lower-level questions in "fast external itemsets" with the minimal appearance are not cleared when the upper-level answer is removed or changed

Steps to reproduce the problem

selectOneExternal.zip

Use case 1:

  1. Fill answers for a minimal appearance
  2. Change answer on first-level - state
  3. Answers for County and City are not cleared

Use case 2:

  1. Fill answers for a minimal appearance
  2. Remove response from first-level - state
  3. Answers for County and City are not cleared

100866015-62ab5100-3498-11eb-9cea-2f0acaa837c8

Expected behavior

Answers for County and City are cleared

created time in 6 hours

PR opened getodk/collect

Fixed: User is able to open two different views from the main screen

Closes #4213

What has been done to verify that this works as intended?

I tested the fix manually and added tests.

Why is this the best possible solution? Were any other approaches considered?

We have custom MultiClickSafeButton so using them in the main menu is better and also fixes the issue since using MultiClickGuard.allowClick in OnClickListener was faulty..

How does this change affect users? Describe intentional changes to behavior and behavior that could have accidentally been affected by code changes. In other words, what are the regression risks?

It's a safe fix so we can just verify that the issue describe in #4213 is fixed.

Do we need any specific form for testing your changes? If so, please attach one.

No.

Does this change require updates to documentation? If so, please file an issue here and include the link below.

No.

Before submitting this PR, please make sure you have:

  • [ ] run ./gradlew checkAll and confirmed all checks still pass OR confirm CircleCI build passes and run ./gradlew connectedDebugAndroidTest locally.
  • [ ] verified that any code or assets from external sources are properly credited in comments and/or in the about file.
  • [ ] verified that any new UI elements use theme colors. UI Components Style guidelines
+93 -50

0 comment

3 changed files

pr created time in 8 hours

pull request commentgetodk/collect

Fixed: Minimal appearance doesn't work with "fast external itemsets"

That whole clearing is faulty it seems to work well if we have a simple cascading select with questions on separate screens but it gets complicated:

  • if we use groups putting answers on one screen
  • if we have additional questions between cascading select levels what's also possible
  • if we wrap particular levels into groups etc.

it might be also dangerous and try to clear what shouldn't be cleared. I think @lognaturel shares my concerns.

If it's the only bug let's file a separate issue to analyze that clearing mechanism and decide what to do.

grzesiek2010

comment created time in 9 hours

PR opened getodk/collect

Add pause/unpause to recording

Closes #4234

What has been done to verify that this works as intended?

Why is this the best possible solution? Were any other approaches considered?

How does this change affect users? Describe intentional changes to behavior and behavior that could have accidentally been affected by code changes. In other words, what are the regression risks?

Do we need any specific form for testing your changes? If so, please attach one.

Does this change require updates to documentation? If so, please file an issue here and include the link below.

Before submitting this PR, please make sure you have:

  • [ ] run ./gradlew checkAll and confirmed all checks still pass OR confirm CircleCI build passes and run ./gradlew connectedDebugAndroidTest locally.
  • [ ] verified that any code or assets from external sources are properly credited in comments and/or in the about file.
  • [ ] verified that any new UI elements use theme colors. UI Components Style guidelines
+1349 -443

0 comment

51 changed files

pr created time in 10 hours

more