Update README.md
Browse files
    	
        README.md
    CHANGED
    
    | @@ -11,7 +11,8 @@ license: apache-2.0 | |
| 11 | 
             
            ---
         | 
| 12 | 
             
            # Model card for DePlot
         | 
| 13 |  | 
| 14 | 
            -
             | 
|  | |
| 15 |  | 
| 16 |  | 
| 17 | 
             
            #  Table of Contents
         | 
| @@ -30,7 +31,25 @@ The abstract of the paper states that: | |
| 30 |  | 
| 31 | 
             
            # Using the model 
         | 
| 32 |  | 
| 33 | 
            -
             | 
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
| 34 |  | 
| 35 | 
             
            You can use the [`convert_pix2struct_checkpoint_to_pytorch.py`](https://github.com/huggingface/transformers/blob/main/src/transformers/models/pix2struct/convert_pix2struct_original_pytorch_to_hf.py) script as follows:
         | 
| 36 | 
             
            ```bash
         | 
| @@ -51,24 +70,6 @@ model.push_to_hub("USERNAME/MODEL_NAME") | |
| 51 | 
             
            processor.push_to_hub("USERNAME/MODEL_NAME")
         | 
| 52 | 
             
            ```
         | 
| 53 |  | 
| 54 | 
            -
            ## Run a prediction
         | 
| 55 | 
            -
             | 
| 56 | 
            -
            You can run a prediction by querying an input image together with a question as follows:
         | 
| 57 | 
            -
            ```python
         | 
| 58 | 
            -
            from transformers import Pix2StructForConditionalGeneration, Pix2StructProcessor
         | 
| 59 | 
            -
            import requests
         | 
| 60 | 
            -
            from PIL import Image
         | 
| 61 | 
            -
             | 
| 62 | 
            -
            model = Pix2StructForConditionalGeneration.from_pretrained('google/deplot')
         | 
| 63 | 
            -
            processor = Pix2StructProcessor.from_pretrained('google/deplot')
         | 
| 64 | 
            -
            url = "https://raw.githubusercontent.com/vis-nlp/ChartQA/main/ChartQA%20Dataset/val/png/5090.png"
         | 
| 65 | 
            -
            image = Image.open(requests.get(url, stream=True).raw)
         | 
| 66 | 
            -
             | 
| 67 | 
            -
            inputs = processor(images=image, text="Generate underlying data table of the figure below:", return_tensors="pt")
         | 
| 68 | 
            -
            predictions = model.generate(**inputs, max_new_tokens=512)
         | 
| 69 | 
            -
            print(processor.decode(predictions[0], skip_special_tokens=True))
         | 
| 70 | 
            -
            ```
         | 
| 71 | 
            -
             | 
| 72 | 
             
            # Contribution
         | 
| 73 |  | 
| 74 | 
             
            This model was originally contributed by Fangyu Liu, Julian Martin Eisenschlos et al. and added to the Hugging Face ecosystem by [Younes Belkada](https://huggingface.co/ybelkada).
         | 
|  | |
| 11 | 
             
            ---
         | 
| 12 | 
             
            # Model card for DePlot
         | 
| 13 |  | 
| 14 | 
            +
            <img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/model_doc/deplot_architecture.png"
         | 
| 15 | 
            +
            alt="drawing" width="600"/>
         | 
| 16 |  | 
| 17 |  | 
| 18 | 
             
            #  Table of Contents
         | 
|  | |
| 31 |  | 
| 32 | 
             
            # Using the model 
         | 
| 33 |  | 
| 34 | 
            +
            You can run a prediction by querying an input image together with a question as follows:
         | 
| 35 | 
            +
             | 
| 36 | 
            +
            ```python
         | 
| 37 | 
            +
            from transformers import Pix2StructProcessor, Pix2StructForConditionalGeneration
         | 
| 38 | 
            +
            import requests
         | 
| 39 | 
            +
            from PIL import Image
         | 
| 40 | 
            +
             | 
| 41 | 
            +
            processor = Pix2StructProcessor.from_pretrained('google/deplot')
         | 
| 42 | 
            +
            model = Pix2StructForConditionalGeneration.from_pretrained('google/deplot')
         | 
| 43 | 
            +
             | 
| 44 | 
            +
            url = "https://raw.githubusercontent.com/vis-nlp/ChartQA/main/ChartQA%20Dataset/val/png/5090.png"
         | 
| 45 | 
            +
            image = Image.open(requests.get(url, stream=True).raw)
         | 
| 46 | 
            +
             | 
| 47 | 
            +
            inputs = processor(images=image, text="Generate underlying data table of the figure below:", return_tensors="pt")
         | 
| 48 | 
            +
            predictions = model.generate(**inputs, max_new_tokens=512)
         | 
| 49 | 
            +
            print(processor.decode(predictions[0], skip_special_tokens=True))
         | 
| 50 | 
            +
            ```
         | 
| 51 | 
            +
             | 
| 52 | 
            +
            # Converting from T5x to huggingface
         | 
| 53 |  | 
| 54 | 
             
            You can use the [`convert_pix2struct_checkpoint_to_pytorch.py`](https://github.com/huggingface/transformers/blob/main/src/transformers/models/pix2struct/convert_pix2struct_original_pytorch_to_hf.py) script as follows:
         | 
| 55 | 
             
            ```bash
         | 
|  | |
| 70 | 
             
            processor.push_to_hub("USERNAME/MODEL_NAME")
         | 
| 71 | 
             
            ```
         | 
| 72 |  | 
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
|  | |
| 73 | 
             
            # Contribution
         | 
| 74 |  | 
| 75 | 
             
            This model was originally contributed by Fangyu Liu, Julian Martin Eisenschlos et al. and added to the Hugging Face ecosystem by [Younes Belkada](https://huggingface.co/ybelkada).
         | 

