Top suggestions for Discrete LanguageExplore more searches like Discrete LanguagePeople interested in Discrete Language also searched for |
Related Products
+with+speech+input+has+enabled+instruction-following+capabilities+for+diverse+speech+tasks.+This+integration+requires+the+use+of+a+speech+encoder%252C+a+speech+adapter%252C+and+an+LLM%252C+trained+on+diverse+tasks.+We+propose+the+use+of+discrete+speech+units+(DSU)%252C+rather+than+continuous-valued+speech+encoder+outputs%252C+that+are+converted+to+the+LLM+token+embedding+space+using+the+speech+adapter.+We+generate+DSU+using+a+self-supervised+speech+encoder+followed+by+k-means+clustering.+The+proposed+model+shows+robust+performance+on+speech+inputs+from+seen%252Funseen+domains+and+instruction-following+capability+in+spoken+question+answering.+We+also+explore+various+types+of+DSU+extracted+from+different+layers+of+the+self-supervised+speech+encoder%252C+as+well+as+Mel+frequency+Cepstral+Coefficients+(MFCC).+Our+findings+suggest+that+the+ASR+task+and+datasets+are+not+crucial+in+instruction-tuning+for+spoken+question+answering+tasks.&cdnurl=https%3a%2f%2fth.bing.com%2fth%2fid%2fR.ff0c515ce372fe473cc531797b943f8e%3frik%3dwIwmY6eXuO5gNQ%26pid%3dImgRaw%26r%3d0&exph=630&expw=1200&q=Discrete+Language&FORM=IRPRST&ck=29F9FAF67B43A4EDF17068D2251FBEBC&selectedIndex=5&itb=0)
%2fcheating--454214895-5a95cc85a9d4f900370d4090.jpg&cdnurl=https%3a%2f%2fth.bing.com%2fth%2fid%2fR.e5108e076797487e8c841fb0e5b8ee23%3frik%3dnsXzpsi4eSOVwA%26pid%3dImgRaw%26r%3d0&exph=4068&expw=5997&q=Discrete+Language&FORM=IRPRST&ck=83C563A1A55DF64F7EB307DECC71C696&selectedIndex=7&itb=0)
Some results have been hidden because they may be inaccessible to you.Show inaccessible results
Feedback
