Upload README.md

README.md

@@ -1,107 +1,42 @@
----
-license: apache-2.0
----
-# Model card for CLAP
+<div align="center">
 
-Model card for CLAP: Contrastive Language-Audio Pretraining
+<img alt="LOGO" src="https://cdn.jsdelivr.net/gh/fishaudio/fish-diffusion@main/images/logo_512x512.png" width="256" height="256" />
 
-![clap_image](https://s3.amazonaws.com/moonup/production/uploads/1678811100805-62441d1d9fdefb55a0b7d12c.png)
+# Bert-VITS2
 
+VITS2 Backbone with multilingual bert
 
-#  Table of Contents
+For quick guide, please refer to `webui_preprocess.py`.
 
-0. [TL;DR](#TL;DR)
-1. [Model Details](#model-details)
-2. [Usage](#usage)
-3. [Uses](#uses)
-4. [Citation](#citation)
+简易教程请参见 `webui_preprocess.py`。
 
-# TL;DR
+## 请注意，本项目核心思路来源于[anyvoiceai/MassTTS](https://github.com/anyvoiceai/MassTTS) 一个非常好的tts项目
+## MassTTS的演示demo为[ai版峰哥锐评峰哥本人,并找回了在金三角失落的腰子](https://www.bilibili.com/video/BV1w24y1c7z9)
 
-The abstract of the paper states that:
+[//]: # (## 本项目与[PlayVoice/vits_chinese]&#40;https://github.com/PlayVoice/vits_chinese&#41; 没有任何关系)
 
-> Contrastive learning has shown remarkable success in the field of multimodal representation learning. In this paper, we propose a pipeline of contrastive language-audio pretraining to develop an audio representation by combining audio data with natural language descriptions. To accomplish this target, we first release LAION-Audio-630K, a large collection of 633,526 audio-text pairs from different data sources. Second, we construct a contrastive language-audio pretraining model by considering different audio encoders and text encoders. We incorporate the feature fusion mechanism and keyword-to-caption augmentation into the model design to further enable the model to process audio inputs of variable lengths and enhance the performance. Third, we perform comprehensive experiments to evaluate our model across three tasks: text-to-audio retrieval, zero-shot audio classification, and supervised audio classification. The results demonstrate that our model achieves superior performance in text-to-audio retrieval task. In audio classification tasks, the model achieves state-of-the-art performance in the zero-shot setting and is able to obtain performance comparable to models' results in the non-zero-shot setting. LAION-Audio-630K and the proposed model are both available to the public.
+[//]: # ()
+[//]: # (本仓库来源于之前朋友分享了ai峰哥的视频，本人被其中的效果惊艳，在自己尝试MassTTS以后发现fs在音质方面与vits有一定差距，并且training的pipeline比vits更复杂，因此按照其思路将bert)
 
+## 成熟的旅行者/开拓者/舰长/博士/sensei/猎魔人/喵喵露/V应当参阅代码自己学习如何训练。
 
-# Usage
+### 严禁将此项目用于一切违反《中华人民共和国宪法》，《中华人民共和国刑法》，《中华人民共和国治安管理处罚法》和《中华人民共和国民法典》之用途。
+### 严禁用于任何政治相关用途。
+#### Video:https://www.bilibili.com/video/BV1hp4y1K78E
+#### Demo:https://www.bilibili.com/video/BV1TF411k78w
+#### QQ Group：815818430
+## References
++ [anyvoiceai/MassTTS](https://github.com/anyvoiceai/MassTTS)
++ [jaywalnut310/vits](https://github.com/jaywalnut310/vits)
++ [p0p4k/vits2_pytorch](https://github.com/p0p4k/vits2_pytorch)
++ [svc-develop-team/so-vits-svc](https://github.com/svc-develop-team/so-vits-svc)
++ [PaddlePaddle/PaddleSpeech](https://github.com/PaddlePaddle/PaddleSpeech)
++ [emotional-vits](https://github.com/innnky/emotional-vits)
++ [fish-speech](https://github.com/fishaudio/fish-speech)
++ [Bert-VITS2-UI](https://github.com/jiangyuxiaoxiao/Bert-VITS2-UI)
+## 感谢所有贡献者作出的努力
+<a href="https://github.com/fishaudio/Bert-VITS2/graphs/contributors" target="_blank">
+  <img src="https://contrib.rocks/image?repo=fishaudio/Bert-VITS2"/>
+</a>
 
-You can use this model for zero shot audio classification or extracting audio and/or textual features.
-
-# Uses
-
-## Perform zero-shot audio classification
-
-### Using `pipeline`
-
-```python
-from datasets import load_dataset
-from transformers import pipeline
-
-dataset = load_dataset("ashraq/esc50")
-audio = dataset["train"]["audio"][-1]["array"]
-
-audio_classifier = pipeline(task="zero-shot-audio-classification", model="laion/clap-htsat-fused")
-output = audio_classifier(audio, candidate_labels=["Sound of a dog", "Sound of vaccum cleaner"])
-print(output)
->>> [{"score": 0.999, "label": "Sound of a dog"}, {"score": 0.001, "label": "Sound of vaccum cleaner"}]
-```
-
-## Run the model:
-
-You can also get the audio and text embeddings using `ClapModel`
-
-### Run the model on CPU:
-
-```python
-from datasets import load_dataset
-from transformers import ClapModel, ClapProcessor
-
-librispeech_dummy = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
-audio_sample = librispeech_dummy[0]
-
-model = ClapModel.from_pretrained("laion/clap-htsat-fused")
-processor = ClapProcessor.from_pretrained("laion/clap-htsat-fused")
-
-inputs = processor(audios=audio_sample["audio"]["array"], return_tensors="pt")
-audio_embed = model.get_audio_features(**inputs)
-```
-
-### Run the model on GPU:
-
-```python
-from datasets import load_dataset
-from transformers import ClapModel, ClapProcessor
-
-librispeech_dummy = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
-audio_sample = librispeech_dummy[0]
-
-model = ClapModel.from_pretrained("laion/clap-htsat-fused").to(0)
-processor = ClapProcessor.from_pretrained("laion/clap-htsat-fused")
-
-inputs = processor(audios=audio_sample["audio"]["array"], return_tensors="pt").to(0)
-audio_embed = model.get_audio_features(**inputs)
-```
-
-
-# Citation
-
-If you are using this model for your work, please consider citing the original paper:
-```
-@misc{https://doi.org/10.48550/arxiv.2211.06687,
-  doi = {10.48550/ARXIV.2211.06687},
-
-  url = {https://arxiv.org/abs/2211.06687},
-
-  author = {Wu, Yusong and Chen, Ke and Zhang, Tianyu and Hui, Yuchen and Berg-Kirkpatrick, Taylor and Dubnov, Shlomo},
-
-  keywords = {Sound (cs.SD), Audio and Speech Processing (eess.AS), FOS: Computer and information sciences, FOS: Computer and information sciences, FOS: Electrical engineering, electronic engineering, information engineering, FOS: Electrical engineering, electronic engineering, information engineering},
-
-  title = {Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation},
-
-  publisher = {arXiv},
-
-  year = {2022},
-
-  copyright = {Creative Commons Attribution 4.0 International}
-}
-```
+[//]: # (# 本项目所有代码引用均已写明，bert部分代码思路来源于[AI峰哥]&#40;https://www.bilibili.com/video/BV1w24y1c7z9&#41;，与[vits_chinese]&#40;https://github.com/PlayVoice/vits_chinese&#41;无任何关系。欢迎各位查阅代码。同时，我们也对该开发者的[碰瓷，乃至开盒开发者的行为]&#40;https://www.bilibili.com/read/cv27101514/&#41;表示强烈谴责。)