Настройки

Укажите год
-

Небесная энциклопедия

Космические корабли и станции, автоматические КА и методы их проектирования, бортовые комплексы управления, системы и средства жизнеобеспечения, особенности технологии производства ракетно-космических систем

Подробнее
-

Мониторинг СМИ

Мониторинг СМИ и социальных сетей. Сканирование интернета, новостных сайтов, специализированных контентных площадок на базе мессенджеров. Гибкие настройки фильтров и первоначальных источников.

Подробнее

Форма поиска

Поддерживает ввод нескольких поисковых фраз (по одной на строку). При поиске обеспечивает поддержку морфологии русского и английского языка
Ведите корректный номера.
Ведите корректный номера.
Ведите корректный номера.
Ведите корректный номера.
Укажите год
Укажите год

Применить Всего найдено 20. Отображено 19.
24-10-2017 дата публикации

Systems and methods for natural language processing for speech content scoring

Номер: US0009799228B2

Computer-implemented systems and methods are provided for scoring content of a spoken response to a prompt. A scoring model is generated for a prompt, where generating the scoring model includes generating a transcript for each of a plurality of training responses to the prompt, dividing the plurality of training responses into clusters based on the transcripts of the training responses, selecting a subset of the training responses in each cluster for scoring, scoring the selected subset of training responses for each cluster, and generating content training vectors using the transcripts from the scored subset. A transcript is generated for a received spoken response to be scored, and a similarity metric is computed between the transcript of the spoken response to be scored and the content training vectors. A score is assigned to the spoken response based on the determined similarity metric.

Подробнее
16-05-2017 дата публикации

Computer-implemented systems and methods for estimating word accuracy for automatic speech recognition

Номер: US0009652999B2

Systems and methods are provided for scoring non-native, spontaneous speech. A spontaneous speech sample is received, where the sample is of spontaneous speech spoken by a non-native speaker. Automatic speech recognition is performed on the sample using an automatic speech recognition system to generate a transcript of the sample, where a speech recognizer metric is determined by the automatic speech recognition system. A word accuracy rate estimate is determined for the transcript of the sample generated by the automatic speech recognition system based on the speech recognizer metric. The spontaneous speech sample is scored using a preferred scoring model when the word accuracy rate estimate satisfies a threshold, and the spontaneous speech sample is scored using an alternate scoring model when the word accuracy rate estimate fails to satisfy the threshold.

Подробнее
20-09-2016 дата публикации

Systems and methods for evaluating difficulty of spoken text

Номер: US0009449522B2

Systems and methods are provided for assigning a difficulty score to a speech sample. Speech recognition is performed on a digitized version of the speech sample using an acoustic model to generate word hypotheses for the speech sample. Time alignment is performed between the speech sample and the word hypotheses to associate the word hypotheses with corresponding sounds of the speech sample. A first difficulty measure is determined based on the word hypotheses, and a second difficulty measure is determined based on acoustic features of the speech sample. A difficulty score for the speech sample is generated based on the first difficulty measure and the second difficulty measure.

Подробнее
16-05-2017 дата публикации

Systems and methods for content scoring of spoken responses

Номер: US0009652991B2

Computer-implemented systems and methods are provided for automatically scoring the content of moderately predictable responses. For example, a computer performing the content scoring analysis can receive a response (either in text or spoken form) to a prompt. The computer can determine the content correctness of the response by analyzing one or more content features. One of the content features is analyzed by applying one or more regular expressions, determined based on training responses associated with the prompt. Another content feature is analyzed by applying one or more context free grammars, determined based on training responses associated with the prompt. Another content feature is analyzed by applying a keyword list, determined based on the test prompt eliciting the response and/or stimulus material. Another content feature is analyzed by applying one or more probabilistic n-gram models, determined based on training responses associated with the prompt. Another content feature is analyzed by comparing a POS response vector, determined based on the response, to one or more POS training vectors, determined based on training responses associated with the prompt. Another content feature is analyzed by comparing a response n-gram count to one or more training n-gram counts using an n-gram matching evaluation metric (e.g., BLEU). Another content feature is analyzed by comparing the response to one to training responses associated with the prompt using a dissimilarity metric (e.g., edit distance and word error rate).

Подробнее
27-03-2018 дата публикации

Systems and methods for generating recitation items

Номер: US9928754B2

Computer-implemented systems and methods are provided for automatically generating recitation items. For example, a computer performing the recitation item generation can receive one or more text sets that each includes one or more texts. The computer can determine a value for each text set using one or more metrics, such as a vocabulary difficulty metric, a syntactic complexity metric, a phoneme distribution metric, a phonetic difficulty metric, and a prosody distribution metric. Then the computer can select a final text set based on the value associated with each text set. The selected final text set can be used as the recitation items for a speaking assessment test.

Подробнее
11-07-2017 дата публикации

Non-scorable response filters for speech scoring systems

Номер: US0009704413B2

A method for scoring non-native speech includes receiving a speech sample spoken by a non-native speaker and performing automatic speech recognition and metric extraction on the speech sample to generate a transcript of the speech sample and a speech metric associated with the speech sample. The method further includes determining whether the speech sample is scorable or non-scorable based upon the transcript and speech metric, where the determination is based on an audio quality of the speech sample, an amount of speech of the speech sample, a degree to which the speech sample is off-topic, whether the speech sample includes speech from an incorrect language, or whether the speech sample includes plagiarized material. When the sample is determined to be non-scorable, an indication of non-scorability is associated with the speech sample. When the sample is determined to be scorable, the sample is provided to a scoring model for scoring.

Подробнее
31-01-2013 дата публикации

Computer-Implemented Systems and Methods for Scoring Concatenated Speech Responses

Номер: US20130030808A1
Принадлежит: EDUCATIONAL TESTING SERVICE

Systems and methods are provided for scoring non-native speech. Two or more speech samples are received, where each of the samples are of speech spoken by a non-native speaker, and where each of the samples are spoken in response to distinct prompts. The two or more samples are concatenated to generate a concatenated response for the non-native speaker, where the concatenated response is based on the two or more speech samples that were elicited using the distinct prompts. A concatenated speech proficiency metric is computed based on the concatenated response, and the concatenated speech proficiency metric is provided to a scoring model, where the scoring model generates a speaking score based on the concatenated speech metric.

Подробнее
06-06-2013 дата публикации

Systems and Methods for Assessment of Non-Native Spontaneous Speech

Номер: US20130144621A1
Принадлежит: EDUCATIONAL TESTING SERVICE

Computer-implemented systems and methods are provided for assessing non-native spontaneous speech pronunciation. Speech recognition on digitized speech is performed using a non-native acoustic model trained with non-native speech to generate word hypotheses for the digitized speech. Time alignment is performed between the digitized speech and the word hypotheses using a reference acoustic model trained with native-quality speech. Statistics are calculated regarding individual words and phonemes in the word hypotheses based on the alignment. A plurality of features for use in assessing pronunciation of the speech are calculated based on the statistics, an assessment score is calculated based on one or more of the calculated features, and the assessment score is stored in a computer-readable memory.

Подробнее
20-06-2013 дата публикации

Computer-Implemented Systems and Methods for Content Scoring of Spoken Responses

Номер: US20130158982A1
Принадлежит: EDUCATIONAL TESTING SERVICE

Systems and methods are provided for scoring a non-scripted speech sample. A system includes one or more data processors and one or more computer-readable mediums. The computer-readable mediums are encoded with a non-scripted speech sample data structure, where the non-scripted speech sample data structure includes: a speech sample identifier that identifies a non-scripted speech sample, a content feature extracted from the non-scripted speech sample, and a content-based speech score for the non-scripted speech sample. The computer-readable mediums further include instructions for commanding the one or more data processors to extract the content feature from a set of words automatically recognized in the non-scripted speech sample and to score the non-scripted speech sample by providing the extracted content feature to a scoring model to generate the content-based speech score. 1. A computer-implemented method of scoring a non-scripted speech sample , comprising:extracting, using a processing system, a content feature from a set of words automatically recognized in the non-scripted speech sample; andscoring, using the processing system, the non-scripted speech sample by providing the extracted content feature to a content scoring model to generate a content-based speech score.2. The method of claim 1 , wherein the content scoring model is built by:transcribing a set of speech samples from a training corpus;assigning a score to each speech sample;partitioning the set of transcribed speech samples of the training corpus into sub-sets, with each of the sub-sets containing speech samples with identical scores; andbuilding score-level training vectors for each sub-set of responses.3. The method of claim 2 , wherein the speech samples of the training corpus are transcribed by human transcribers.4. The method of claim 2 , wherein the speech samples of the training corpus are transcribed by an automated speech recognizer.5. The method of claim 2 , wherein the score for each ...

Подробнее
22-05-2014 дата публикации

Systems and Methods for Evaluating Difficulty of Spoken Text

Номер: US20140141392A1
Принадлежит: EDUCATIONAL TESTING SERVICE

Systems and methods are provided for assigning a difficulty score to a speech sample. Speech recognition is performed on a digitized version of the speech sample using an acoustic model to generate word hypotheses for the speech sample. Time alignment is performed between the speech sample and the word hypotheses to associate the word hypotheses with corresponding sounds of the speech sample. A first difficulty measure is determined based on the word hypotheses, and a second difficulty measure is determined based on acoustic features of the speech sample. A difficulty score for the speech sample is generated based on the first difficulty measure and the second difficulty measure. 1. A computer-implemented method of assigning a difficulty score to a speech sample , comprising:performing speech recognition of the speech sample to generate word hypotheses for the speech sample;performing time alignment between the speech sample and the word hypotheses to associate the word hypotheses with corresponding sounds of the speech sample;determining a first difficulty measure based on the word hypotheses for the speech sample;determining a second difficulty measure based on acoustic features of the speech sample;generating a difficulty score for the speech sample based on the first difficulty measure and the second difficulty measure; andstoring the difficulty score in a computer-readable memory.2. The method of claim 1 , wherein the second difficulty measure is based on the acoustic features of the speech sample and the word hypotheses.3. The method of claim 1 , wherein the second difficulty measure is based on pronunciation quality in the speech sample.4. The method of claim 1 , wherein the second difficulty measure is based on prosody of the speech sample.5. The method of claim 1 , wherein the first difficulty measure is a vocabulary measure claim 1 , a grammar measure claim 1 , or a discourse measure.6. The method of claim 1 , wherein the speech sample includes speech by ...

Подробнее
17-07-2014 дата публикации

Systems and Methods for Natural Language Processing for Speech Content Scoring

Номер: US20140199676A1
Принадлежит: EDUCATIONAL TESTING SERVICE

Computer-implemented systems and methods are provided for scoring content of a spoken response to a prompt. A scoring model is generated for a prompt, where generating the scoring model includes generating a transcript for each of a plurality of training responses to the prompt, dividing the plurality of training responses into clusters based on the transcripts of the training responses, selecting a subset of the training responses in each cluster for scoring, scoring the selected subset of training responses for each cluster, and generating content training vectors using the transcripts from the scored subset. A transcript is generated for a received spoken response to be scored, and a similarity metric is computed between the transcript of the spoken response to be scored and the content training vectors. A score is assigned to the spoken response based on the determined similarity metric. 1. A computer-implemented method of scoring content of a spoken response to a prompt , comprising: generating a transcript for each of a plurality of training responses to the prompt;', 'dividing the plurality of training responses into clusters based on the transcripts of the training responses;', 'selecting a subset of the training responses in each cluster for scoring;', 'scoring the selected subset of training responses for each cluster; and', 'generating content training vectors using the transcripts from the scored subset;, 'generating a scoring model for a prompt, wherein generating the scoring model comprisesgenerating transcript for a received spoken response to be scored;computing a similarity metric between the transcript of the spoken response to be scored and the content training vectors; andassigning a score to the spoken response based on the similarity metric.2. The method of claim 1 , wherein generating the scoring model for the prompt further comprises:determining a score for each cluster based on scores for the selected subset for each cluster; andwherein the ...

Подробнее
11-09-2014 дата публикации

Systems and Methods for Content Scoring of Spoken Responses

Номер: US20140255886A1
Принадлежит: EDUCATIONAL TESTING SERVICE

Computer-implemented systems and methods are provided for automatically scoring the content of moderately predictable responses. For example, a computer performing the content scoring analysis can receive a response (either in text or spoken form) to a prompt. The computer can determine the content correctness of the response by analyzing one or more content features. One of the content features is analyzed by applying one or more regular expressions, determined based on training responses associated with the prompt. Another content feature is analyzed by applying one or more context free grammars, determined based on training responses associated with the prompt. Another content feature is analyzed by applying a keyword list, determined based on the test prompt eliciting the response and/or stimulus material. Another content feature is analyzed by applying one or more probabilistic n-gram models, determined based on training responses associated with the prompt. Another content feature is analyzed by comparing a POS response vector, determined based on the response, to one or more POS training vectors, determined based on training responses associated with the prompt. Another content feature is analyzed by comparing a response n-gram count to one or more training n-gram counts using an n-gram matching evaluation metric (e.g., BLEU). Another content feature is analyzed by comparing the response to one to training responses associated with the prompt using a dissimilarity metric (e.g., edit distance and word error rate).

Подробнее
09-07-2015 дата публикации

Non-Scorable Response Filters for Speech Scoring Systems

Номер: US20150194147A1
Принадлежит:

A method for scoring non-native speech includes receiving a speech sample spoken by a non-native speaker and performing automatic speech recognition and metric extraction on the speech sample to generate a transcript of the speech sample and a speech metric associated with the speech sample. The method further includes determining whether the speech sample is scorable or non-scorable based upon the transcript and speech metric, where the determination is based on an audio quality of the speech sample, an amount of speech of the speech sample, a degree to which the speech sample is off-topic, whether the speech sample includes speech from an incorrect language, or whether the speech sample includes plagiarized material. When the sample is determined to be non-scorable, an indication of non-scorability is associated with the speech sample. When the sample is determined to be scorable, the sample is provided to a scoring model for scoring. 1. A computer-implemented method of scoring non-native speech , comprising:receiving a speech sample spoken by a non-native speaker to be scored by a scoring model using a processing system, wherein the scoring model generates scores for speech samples based on one or more speech metrics;performing automatic speech recognition and metric extraction on the speech sample to generate a transcript of the speech sample and a confidence score associated with one of the scoring model speech metrics using the processing system;determining whether the speech sample is scorable or non-scorable based upon the transcript and the confidence score using the processing system;associating an indication of non-scorability with the speech sample when the sample is determined to be non-scorable using the processing system; andproviding the sample to a scoring model for scoring when the sample is determined to be scorable using the processing system.2. The method of claim 1 , further comprising:identifying a non-speech metric associated with a speaker ...

Подробнее
18-09-2014 дата публикации

Systems and Methods for Generating Recitation Items

Номер: US20140278376A1
Принадлежит: EDUCATIONAL TESTING SERVICE

Computer-implemented systems and methods are provided for automatically generating recitation items. For example, a computer performing the recitation item generation can receive one or more text sets that each includes one or more texts. The computer can determine a value for each text set using one or more metrics, such as a vocabulary difficulty metric, a syntactic complexity metric, a phoneme distribution metric, a phonetic difficulty metric, and a prosody distribution metric. Then the computer can select a final text set based on the value associated with each text set. The selected final text set can be used as the recitation items for a speaking assessment test.

Подробнее
02-10-2014 дата публикации

Systems and Methods for Automated Scoring of Spoken Language in Multiparty Conversations

Номер: US20140297277A1
Принадлежит: EDUCATIONAL TESTING SERVICE

Systems and methods are provided for scoring spoken language in multiparty conversations. A computer receives a conversation between an examinee and at least one interlocutor. The computer selects a portion of the conversation. The portion includes one or more examinee utterances and one or more interlocutor utterances. The computer assesses the portion using one or more metrics, such as: a pragmatic metric for measuring a pragmatic fit of the one or more examinee utterances; a speech act metric for measuring a speech act appropriateness of the one or more examinee utterances; a speech register metric for measuring a speech register appropriateness of the one or more examinee utterances; and an accommodation metric for measuring a level of accommodation of the one or more examinee utterances. The computer computes a final score for the portion of the conversation based on the one or more metrics applied. 1. A computer-implemented method of assessing communicative competence , the method comprising:receiving a conversation between an examinee and at least one interlocutor;selecting a portion of the conversation, wherein the portion includes one or more examinee utterances and one or more interlocutor utterances; pragmatic metric for measuring a pragmatic fit of the one or more examinee utterances;', 'speech act metric for measuring a speech act appropriateness of the one or more examinee utterances;', 'speech register metric for measuring a speech register appropriateness of the one or more examinee utterances; and', 'accommodation metric for measuring a level of accommodation of the one or more examinee utterances;, 'assessing the portion using one or more metrics selected from the group consisting ofcomputing a final score for the portion of the conversation based on at least the one or more metrics applied.2. The method of claim 1 , wherein the conversation is in audio format claim 1 , the method further comprising:converting the conversation into text format.3. ...

Подробнее
03-09-2015 дата публикации

Deep Convolutional Neural Networks for Automated Scoring of Constructed Responses

Номер: US20150248608A1
Принадлежит:

Systems and methods are provided for automatically scoring a constructed response. The constructed response is processed to generate a plurality of numerical vectors that is representative of the constructed response. A model is applied to the plurality of numerical vectors. The model includes an input layer configured to receive the plurality of numerical vectors, the input layer being connected to a following layer of the model via a first plurality of connections. Each of the connections has a first weight. An intermediate layer of nodes is configured to receive inputs from an immediately-preceding layer of the model via a second plurality of connections, each of the connections having a second weight. An output layer is connected to the intermediate layer via a third plurality of connections, each of the connections having a third weight. The output layer is configured to generate a score for the constructed response. 1. A computer-implemented method of constructing a model to automatically score a constructed response , the method comprising: an input layer configured to receive a plurality of numerical vectors that is representative of a constructed response, the input layer being connected to a following layer of the model via a first plurality of connections, each of the connections having an associated first weight and passing a portion of the plurality of numerical vectors to the following layer,', 'a first intermediate layer of nodes configured to receive inputs from an immediately-preceding layer of the model via a second plurality of connections, each of the second plurality of connections having an associated second weight, wherein each node of the first intermediate layer generates an output based on a weighted summation of received inputs, and', 'an output layer connected to the first intermediate layer via a third plurality of connections, each of the third plurality of connections having an associated third weight and passing one of the outputs ...

Подробнее
22-12-2015 дата публикации

Computer-implemented systems and methods for content scoring of spoken responses

Номер: US9218339B2
Принадлежит: EDUCATIONAL TESTING SERVICE

Systems and methods are provided for scoring a non-scripted speech sample. A system includes one or more data processors and one or more computer-readable mediums. The computer-readable mediums are encoded with a non-scripted speech sample data structure, where the non-scripted speech sample data structure includes: a speech sample identifier that identifies a non-scripted speech sample, a content feature extracted from the non-scripted speech sample, and a content-based speech score for the non-scripted speech sample. The computer-readable mediums further include instructions for commanding the one or more data processors to extract the content feature from a set of words automatically recognized in the non-scripted speech sample and to score the non-scripted speech sample by providing the extracted content feature to a scoring model to generate the content-based speech score.

Подробнее
26-12-2023 дата публикации

Automated content feedback generation system for non-native spontaneous speech

Номер: US11854530B1
Принадлежит: EDUCATIONAL TESTING SERVICE

An electronic audio file is received that comprises spontaneous speech responsive to a prompt in a non-native language of a speaker. Thereafter, the electronic audio file is parsed into a plurality of spoken words. The spoken words are then normalized to remove stop words and disfluencies. At least one trained content scoring model is then used to determine an absence of pre-defined key points associated with the prompt in the normalized spoken words. A list of the determined absent key points can be generated. This list can then be displayed/caused to be displayed in a graphical user interface along with feedback to improve content completeness. Related apparatus, systems, techniques and articles are also described.

Подробнее
01-05-2014 дата публикации

Non-scorable response filters for speech scoring systems

Номер: WO2012134997A3
Принадлежит: EDUCATIONAL TESTING SERVICE

A method for scoring non-native speech includes receiving a speech sample spoken by a non-native speaker and performing automatic speech recognition and metric extraction on the speech sample to generate a transcript of the speech sample and a speech metric associated with the speech sample. The method further includes determining whether the speech sample is scorable or non-scorable based upon the transcript and speech metric, where the determination is based on an audio quality of the speech sample, an amount of speech of the speech sample, a degree to which the speech sample is off-topic, whether the speech sample includes speech from an incorrect language, or whether the speech sample includes plagiarized material. When the sample is determined to be non-scorable, an indication of non-scorability is associated with the speech sample. When the sample is determined to be scorable, the sample is provided to a scoring model for scoring.

Подробнее