* Split long functions and optimize code
* Replace `errors.Wrap()` with the `fmt.Errorf()`; rename `process*()` functions to `parse*()`
---------
Co-authored-by: Konstantin Koval
Updated DB images in tests to align with the PROD setup
Try to upload coverage only if tests were executed (passed or failed) - not skipped or canceled
Added ESLint execution after UI tests with results exported to artifact
Shifted scheduled run time from 00 minutes according to GitHub recommendations, as the 00 minutes of each hour is the busiest time
Dynamically extract a list of langs from the repo to use them for CodeQL analysis instead of hardcoded ones
Updated versions of several outdated actions
Added the Autobuild step before CodeQL analysis for GO
Added the Anchore dependency scan job, reporting to the Security tab. I can add steps to manage PR comments with the results, but I need a token to be provided by @viktorstrate
Added the Hadolint Dockerfile scan job, reporting to the Security tab. I can add steps to manage PR comments with the results, but I need a token to be provided by @viktorstrate
Implemented weekly rebuild of images for the latest commit in the master branch and the latest released tag. It will recreate images with the recent base image and 3rd-party dependencies even if there were no new pushes for a long time
Added the Dockle container analysis job to be run on master and tag and validate just pushed images, reporting to the Security tab
Added golangci-lint config to the /api folder, as a starting point and for local usage
Added 2 weekly jobs for Dependabot:
-- Maintain dependencies for GitHub Actions
-- Maintain dependencies for Dockerfile
---------
Co-authored-by: Konstantin Koval
* names, constants of reusable strings, removed unneeded `if`, replaced deprecated imports
* more deprecated imports replaced
* What is this?))
* Fix a path and quote a var in Dockerfile
* Addressing review comments
---------
Co-authored-by: Konstantin Koval <kkb@ukr.net>
Once you have more than a few paths assigned to a user, it becomes
quite hard to oversee the list.
Personally, I would add my albums to Photoview one by one, so I can
manage the processing better. For example, I can review and fix the face
recognition for one new album at a time, and I don't need to deal with
too much data.
Since I started to add the albums in a random order, it's relatively
hard to find which album is added and which is not.
An easy fix is to order the photo paths by their name.
- Extended the query of the rootAlbums to order them by path
* Validate incoming GPS data and throw an error if it is incorrect, storing Null values
* Extracted GPS data processing to function in external parser; optimized IF in internal parser; removed unnecessary comments; set exact values for positive test
* Add the migration for removing existing invalid GPS data and its test; added better errors to asserts in the GPS validation test
* Install FFmpeg and ExifTool on the API unit-test environment
* Fix 'stripped.jpg', 'IncorrectGPS.jpg', and 'CorrectGPS.jpg' tests for the external parser
* Optimized data validation in the external parser, returned error by the internal parser for invalid data, updated test to expect errors and handle them
* Switched from error to log entry in case of incorrect GPS data, as error handling is not so transparent in the internal parser
---------
Co-authored-by: Konstantin Koval
* Bump go version to 1.22. Bump all go module to the newest version, except pinning libheif@v1.15.1 to match the c lib version in Debian bookworm.
* Update go_wrapper.sh to fit golang image.
* Update the way to set go envs.
* Test against go 1.22
* Set default shell for all building stage in Dockerfile.
Add a "no_face_detection" build tag to disable face detection
when building. This is useful when installing the face detection
dependencies is undesirable and cuts down build times (e.g. on a
Raspberry Pi).
I encountered with the following error:
> 2023/02/05 07:33:00 /app/scanner/face_detection/face_detector.go:92 sql: transaction has already been committed or rolled back
> [0.042ms] [rows:0] SELECT * FROM `media` WHERE `media`.`id` = 823 ORDER BY `media`.`id` LIMIT 1
> 2023/02/05 07:33:00 ERROR: Error detecting faces in image (/photos/Borzsony2017/DSC_0028.NEF): sql: transaction has already been committed or rolled back
It turned out it comes from the api/routes/photos.go
I found a very similar code in album_scanner.go.
The difference I saw was that while in the single photo request the
transaction passed to the `scanner_tasks.Tasks.BeforeProcessMedia` call,
in the album_scann.go the transaction created after this call and
created from the context which returned by `BeforeProcessMedia`.
Another difference was that in the `ProcessSingleMedia` call the
`AfterProcessMedia` call was called with the same - db transaction -
context, in the album_scanner it was called outside of the transaction.
I changed the logic by merging the two behavior:
Create the transaction from the context of `BeforeProcessMedia` and also
use the transaction context in the `AfterProcessMedia`.
After the change the error disappeared.
So to have it in a common place I extracted that logic into a function
and use for both the single photo request and in the album scanner.
I did not go more deeper to find out what's going on with the context
under the hood.
