In various industries such as modern healthcare, financial services, and public sectors, many AWS customers store billions of images or PDF documents using Amazon Simple Storage Service (Amazon S3). However, until now, these customers have been unable to effectively extract information from these documents.
How to Solve Bulk Document Processing Issues with AWS Textract?
AWS provides a solution called Intelligent Document Processing (IDP), which leverages artificial intelligence services such as Amazon Textract. This enables customers to utilize machine learning technology to quickly and accurately process data from PDFs or document images. With this solution, you can extract text from documents, refine models, aggregate data, or send it to databases.
This article will introduce two solutions for processing large volumes of documents into raw text files and storing them in Amazon S3. The first method uses Python scripts, allowing them to be run from any server or instance, making it the fastest way to get started. The second method involves using the AWS Cloud Development Kit (AWS CDK) to construct key-deployed integrations with various infrastructure components. AWS CDK provides a flexible and adaptable framework for handling documents, building end-to-end IDP pipelines, and extending functionality to meet specific business needs.
Solution 1: Using Python Scripts
This solution utilizes Amazon Textract to process raw text documents quickly and is designed to resume from the point of interruption in case of script failure. The script leverages three different services: Amazon S3, Amazon DynamoDB, and Amazon Textract. In this solution, we create a DynamoDB table to store object references for documents in Amazon S3. The script enumerates this list and uses the Amazon Textract asynchronous API to maximize service throughput.
The diagram below illustrates the event sequence in the script. Upon completion, the status and time used will be returned to the SageMaker studio console.
Solution 2: Using Serverless AWS CDK Constructs
This solution uses AWS Step Functions and Lambda functions to orchestrate the IDP pipeline. We use the IDP AWS CDK constructs, leveraging Step Functions to iteratively map through all files in the S3 bucket. This solution also includes two Lambda functions for parsing and storing text extracted from Amazon Textract.
The diagram below illustrates the Step Functions workflow.
Regardless of which solution you choose, you will be able to quickly process millions of pages of documents. Before running the solution in production, it is recommended to test it with a subset of documents to ensure the results meet expectations.
In conclusion, these solutions enable customers to easily convert large volumes of documents into text data usable for artificial intelligence and search purposes.
Source：https://aws.amazon.com/tw/blogs/machine-learning/create-a-document-lake-using-large-scale-text-extraction-from-documents-with-amazon-textract/