[v1.1.1] Enhancements and Bug Fixes

Description

This pull request includes several updates and improvements to the E2M (Everything to Markdown) project. The changes are aimed at enhancing performance, refining documentation, and fixing identified issues.

Key Changes

• Refactoring: Merged doc and docx converters for streamlined functionality.
• Performance Improvements: Optimized API performance and updated dependencies for better reliability.
• Documentation: Added, updated, and reformatted documentation for clearer guidance.
• Testing: Introduced new test cases to improve code coverage and ensure stability.
• Build Process: Enhanced the build process with fixes to the workflow and the addition of necessary utilities.

Commits

• 🦄 refactor: merge doc and docx converter
• 🎈 perf: api
• 📃 docs: add community
• 🔧 build: update dependencies
• 🧪 test: add test cases
• 🦄 refactor: create file utils
• 🦄 refactor: format
• 🔧 build: fix workflow
• 🔧 build: use sudo
• 🔧 build: add poppler-utils in docker
• 🔧 build: add dependencies
• 📃 docs: update version to v1.1.1
• 🔧 build: add built docker
• 📃 docs: update docker start info

Version Update

The project version has been updated to v1.1.1.

Docker

• Added poppler-utils to Docker for enhanced functionality.
• Updated Docker start information in documentation.

Testing

• New test cases have been added to ensure the recent changes do not introduce any regressions.

Dependencies

• Updated project dependencies to the latest versions for security and compatibility.

Reviewers

• @Jing-yilin

Labels

• Bug Fix
• Enhancement
• Documentation

Projects

• E2M Development

Milestone

• E2M v1.1.1 Release

Additional Notes

• The community guidelines have been updated in the documentation.
• The Docker repository has been updated with the latest images.

[v1.1.0] LLM Empowered and Bug Fixes

Title: Version 1.1.0 Release: Major Enhancements and Bug Fixes

Commits and Changes:

Commits on May 31, 2024

• 🎈 perf: avoid reloading model
  • Improved performance by avoiding unnecessary reloading of models.

Commits on Jun 1, 2024

• 🎈 perf: rm txt_converter

  • Removed the obsolete txt_converter to streamline the codebase.

• ✨ feat: add markdown preview

• Added functionality to preview markdown content, enhancing the user
  experience.

• 🦄 refactor: better structure

• Refactored the project structure for better organization and
  maintainability.

• 🎈 perf: select pages

• Improved performance by allowing the selection of specific pages for
  processing.

• 🧪 test: add docker check workflow

• Added a Docker check workflow to ensure consistent and reliable Docker
  setups.

• 🔧 build: update requirements

• Updated project requirements to include necessary dependencies and
  remove redundant ones.

• 🐳 chore: better setup_db_docker.sh

• Improved the setup_db_docker.sh script for a more efficient database
  setup.

• 🎈 perf: add USE_DB choices

• Added options for using different databases, enhancing flexibility and
  configurability.

• 🐞 fix: docker-compose file bug

• Fixed bugs in the docker-compose file to ensure smooth container
  orchestration.

Commits on Jun 3, 2024

• ✨ feat: md parser

• Implemented a Markdown parser for converting various file formats to
  markdown.

• ✨ feat: llm feature

• Added features for utilizing large language models (LLMs) for advanced
  text processing.

• 🎈 perf: json tree to frontend

• Enhanced the frontend with a JSON tree preview for better
  visualization of JSON responses.

• 🔧 build: update dependencies

• Updated dependencies to the latest versions for improved stability and
  security.

• 📃 docs: new api doc

• Created new API documentation to provide clear and comprehensive
  guidance for users.

• 📃 docs: show 2json and 2md

• Updated documentation to demonstrate the new 2json and 2md conversion
  features.

• 📃 docs: update version to v1.1.0

  • Updated versioning to reflect the new release, now at version 1.1.0.

• 🔧 build: update dependencies

• Further updated dependencies to ensure all necessary libraries are
  included.

• 🔧 build: change default converter

• Changed the default converter for better default behavior and user
  experience.

• 📃 docs: update intro

• Updated the project introduction in the documentation to reflect the
  latest changes and improvements.

Summary of Key Features:

• Markdown and JSON Format Conversion: Users can now convert files
  to both markdown and JSON formats, with customizable JSON output
  formats.
• Enhanced API Response Structure: The API response structure has
  been highly refined, providing more detailed and structured JSON data.
• Frontend JSON Tree Preview: Implemented a JSON tree preview on the
  frontend for a more intuitive visualization of JSON responses.
• Bug Fixes: Addressed and fixed numerous bugs to improve overall
  stability and performance.

This update significantly enhances the functionality and usability of
the E2M project, making it more robust and user-friendly.

Testing:

• All changes have been tested locally and through the updated Docker
  check workflow.
• Comprehensive unit tests have been added to ensure the new features
  work as expected.

Documentation:

• The documentation has been updated to reflect the new features and
  changes.
• Examples and usage guides for the new 2json and 2md features have been
  included.

---

Please review the changes and provide feedback. Thank you for your
contributions and support!

---

Add Web

Add Web

[v1.0.5] - 2024-05-31

🆕 New Features

• Feature 1: Support md, txt, doc, docx, pdf, py, json, yaml, yml
• Feature 2: Add frontend web at http://localhost:3000

---

If you have any questions or feedback, please submit them via
issues.

Thank you for your support!

Docker GPU Feature

Docker GPU Feature

[v1.0.4] - 2024-05-29

🆕 New Features

• Feature 1: Support GPU Acceleration in Docker

🛠️ Fixes

• Bug 1: Docker init bug
• Bug 2: pdf convert bug

---

If you have any questions or feedback, please submit them via
issues.

Thank you for your support!

Add Demo For Instruction

Add Demo For Instruction

[v1.0.3] - 2024-05-28

🆕 New Features

• Feature 1: Add Demo
  • Related links or documentation: Demo
• Feature 2: Add params: langs

🛠️ Fixes

• Bug 1: Cache error fixed

Add PDF Converter

Release Title

[v1.0.2] - 2024-05-26

🆕 New Features

• Feature 1: Use postgresql as cache.
• Feature 2: Use Surya for PDF detection.

The MVP Version

📋 Description

This Version has complete a minimum runnable version of the E2M project.

✨ Changes

1. Add classes: Converters, Parsers.
2. Use env variables to control production version.
3. Add Swagger UI for API documentation.

🛠 How to Test

Checkout to this branch: git checkout main
Install dependencies: cd app && pip install -r requirements.txt
Run the application: flask run --host 0.0.0.0 --port=8765

Visit http://127.0.0.1:8765/swagger/ to view Swagger UI.
Test the endpoint http://127.0.0.1:8765/api/v1/ping to ensure it returns the expected response.

📸 Screenshots (if applicable)

✅ Checklist

Code is working as expected
Tests have been added/updated (if applicable)
Documentation has been updated (if applicable)
All checks have passed (CI/CD)