* Refactored code; temp disable workflows on push
* remove printf leftovers
* A better sanitizing code
* extend auth routes tests
* more tests, fix 2 bugs in token processing
* more refactoring and fixes
* optimize video mediaURLs DB query and make testCachePath in utils thread-safe
* fix always passing test
* and more refactoring; 1 more test, but cleanup still has to be fixed
* more debug code in the test
* fix the test
* Final commit
* Adding a defer function just in case of panic
* Address some of review comments
* Protect the shared var by mutex; implement correct context-aware scanning
* Better error; better initial func wrap/capture; better log messages in `photos.go`;
* Addressing a few more review comments
* Order the query results; a better error type catch; setting the MP4 content type header explicitly
* try to fix the cancelation detection condition
* Extract a function and call it by name
---------
Co-authored-by: Konstantin Koval
* Initial fix of a race condition
* better logging
* some refactoring and graceful shutdown
* fix more potential race condition places; a better app shutdown
* add test and fix some issues
* some housekeeping
* more descriptive asserts
* Address review suggestions
* Better server shutdown logic + better message about the .env file not found
* Create context inside the shutdown goroutine; removed unnecessary shutdown from the main function
* Move HTTP server to the main process and remove unnecessary code
* log shutdown error if any
* Get back to the local `scanner` variable
* Protecting scanner with mutex
* Adding the mutex to the shutdown function
* Addressing the race condition at the init stage
---------
Co-authored-by: Konstantin Koval
* Add MagickWand perf test.
* Use MagickWand.
* Calculate the right thumbnail dimension.
* Remove magick bin file.
* Add building dependencies.
* Build images.
* Add --no-install-recommends
* Remove unused field.
* Fix typo
* Use uint in the worker.
* Add guard
* Always update apt.
* Fix EncodeJpeg() with the quality
* Rename the local error with a better name.
* Add worker.Terminate()
* Fix typo
* Rebase with master and rollback the build workflow.
* Handle media not found issue
* temporary run workflows for master pushes only; tests
* revert temp workflow changes
---------
Co-authored-by: Konstantin Koval
* Use libmagic to identify file types.
* Fix mime array for database querying.
* go mod tidy
* Fix typo
* Another fix for the scheduled image build workflow for tag case (#1137)
* Try to fix the incorrect tag processing
* more debug code
* try with context=git
* hopefully, final debug
* the final commit with no debug code
---------
Co-authored-by: Konstantin Koval
* Add benchmark.
* rollback workflows.
* Fix AI comments.
* Build image.
* Fix AI comments.
* Fix AI comments.
* Rollback build workflow.
---------
Co-authored-by: Kostiantyn <32730812+kkovaletp@users.noreply.github.com>
* Add log package. Update worker with new log package.
* Add init error to worker.
* Init workers when binary launching.
* Add tests to cover different errors.
* names, constants of reusable strings, removed unneeded `if`, replaced deprecated imports
* more deprecated imports replaced
* What is this?))
* Fix a path and quote a var in Dockerfile
* Addressing review comments
---------
Co-authored-by: Konstantin Koval <kkb@ukr.net>
Once you have more than a few paths assigned to a user, it becomes
quite hard to oversee the list.
Personally, I would add my albums to Photoview one by one, so I can
manage the processing better. For example, I can review and fix the face
recognition for one new album at a time, and I don't need to deal with
too much data.
Since I started to add the albums in a random order, it's relatively
hard to find which album is added and which is not.
An easy fix is to order the photo paths by their name.
- Extended the query of the rootAlbums to order them by path
* Validate incoming GPS data and throw an error if it is incorrect, storing Null values
* Extracted GPS data processing to function in external parser; optimized IF in internal parser; removed unnecessary comments; set exact values for positive test
* Add the migration for removing existing invalid GPS data and its test; added better errors to asserts in the GPS validation test
* Install FFmpeg and ExifTool on the API unit-test environment
* Fix 'stripped.jpg', 'IncorrectGPS.jpg', and 'CorrectGPS.jpg' tests for the external parser
* Optimized data validation in the external parser, returned error by the internal parser for invalid data, updated test to expect errors and handle them
* Switched from error to log entry in case of incorrect GPS data, as error handling is not so transparent in the internal parser
---------
Co-authored-by: Konstantin Koval
Add a "no_face_detection" build tag to disable face detection
when building. This is useful when installing the face detection
dependencies is undesirable and cuts down build times (e.g. on a
Raspberry Pi).
I encountered with the following error:
> 2023/02/05 07:33:00 /app/scanner/face_detection/face_detector.go:92 sql: transaction has already been committed or rolled back
> [0.042ms] [rows:0] SELECT * FROM `media` WHERE `media`.`id` = 823 ORDER BY `media`.`id` LIMIT 1
> 2023/02/05 07:33:00 ERROR: Error detecting faces in image (/photos/Borzsony2017/DSC_0028.NEF): sql: transaction has already been committed or rolled back
It turned out it comes from the api/routes/photos.go
I found a very similar code in album_scanner.go.
The difference I saw was that while in the single photo request the
transaction passed to the `scanner_tasks.Tasks.BeforeProcessMedia` call,
in the album_scann.go the transaction created after this call and
created from the context which returned by `BeforeProcessMedia`.
Another difference was that in the `ProcessSingleMedia` call the
`AfterProcessMedia` call was called with the same - db transaction -
context, in the album_scanner it was called outside of the transaction.
I changed the logic by merging the two behavior:
Create the transaction from the context of `BeforeProcessMedia` and also
use the transaction context in the `AfterProcessMedia`.
After the change the error disappeared.
So to have it in a common place I extracted that logic into a function
and use for both the single photo request and in the album scanner.
I did not go more deeper to find out what's going on with the context
under the hood.
2.3.12 -> 2.3.13 restructured ScanAlbum so that errors were returned rather than raising a ScannerError directly - this ensures that the errors are logged correctly.
