PUNE: A study investigating why the same artificial-intelligence acceleration technique can more than double performance on one workload while slowing another has received the Best Paper Award at the International Conference on Emerging in Computing Technologies (INCECT 2026), hosted by Jain College of Engineering in Belagavi, Karnataka, and held on August 21-22, 2026, a statement issued by the college said.The paper, “Not All Domains Speculate Equally: Characterizing and Cheaply Estimating Acceptance-Rate Variation in Speculative Decoding,” was authored by Varun Kotte, Rohit Joshi, Supratim Dutta and Ravindra Rajasekhar Kavuru.According to the conference organisers, INCECT 2026 received 4,543 paper submissions, of which 232 papers, approximately 5.1%, were selected following the conference’s peer-review process. The study was subsequently selected for the Best Paper Award by an award jury chaired by Dr. Krupa Rasane, a General Chair of INCECT 2026.The study combines rigorous cross-domain evidence, a clear demonstration that acceleration can backfire, and a practical method that teams can use before committing resources to deployment.Large language models generate text one token at a time, with each token requiring an expensive pass through the model. Speculative decoding seeks to reduce this cost by having a smaller “draft” system propose several future tokens, which the larger target model verifies in parallel. When the draft predicts accurately, several tokens are accepted at once and generation speeds up. When too many are rejected, the additional work can cancel the benefit or make generation slower.Evaluations of speculative decoding typically report a single average speedup, usually measured on English conversational prompts. The winning study examined whether such a number could be safely applied to other workloads.The best individual feature achieved a Spearman correlation of 0.25, while a regression using all the features reached an R-squared value of 0.19. A model using only the domain label performed markedly better, reaching an R-squared value of 0.47.Acceptance, the authors conclude, is driven primarily by the continuation the model generates rather than by readily visible characteristics of the input.A 16-prompt test before deployment The paper’s practical contribution is a lightweight offline probe. The target model first generates a short continuation, after which a candidate drafter is scored against it token by token. A parameter-free formula then converts that agreement into a predicted online acceptance rate.At the domain level, the probe’s estimated online acceptance rates had a mean absolute error of 0.013 relative to the measured rates. A 16-prompt version, requiring approximately 37 seconds on one A100 GPU, made the correct enable-or-not and which-drafter decision for all five tested domains.Because it uses ordinary model inference rather than a speculative scheduler, the probe can run before a specialised serving integration exists. The researchers applied it to a Gemma-3 model pair for which their vLLM build could not run online draft-model speculation because of a hybrid-attention limitation.The probe’s relative domain ordering matched the Llama-3 results, while separate online n-gram tests confirmed that domain-dependent variation persisted across both model families. Kotte, the paper’s first author and an independent researcher based in San Jose, California, said the study grew out of a practical deployment question.“Speculative decoding is sometimes discussed as though it were a switch that makes every workload faster,” Kotte said. “Our results show that the outcome depends heavily on both the domain and the drafter. Testing a small sample from the intended workload can reveal that difference before a team spends significant time building a full serving integration.”The authors said Kotte originated the research question and led the experimental design, implementation, analysis and initial manuscript preparation, while Joshi, Dutta and Kavuru contributed to methodology, validation, interpretation and manuscript review. Joshi is listed in the paper with an affiliation to Liberty Mutual Insurance, while Dutta is listed with an affiliation to T-Mobile. Kavuru is an independent researcher based in Everett, Washington.In discussing the study’s implications, Joshi highlighted the importance of evaluating acceleration claims on the workloads a system will actually serve. Dutta pointed to the 16-prompt probe as a practical way for teams to compare drafting strategies before committing engineering resources to a full serving integration. Kavuru noted that the cross-model tests reinforced the finding that domain-dependent performance was not confined to a single model configuration.The authors caution that the results cover two model families, five domains, greedy decoding and a batch-size-one latency setting. Absolute speedups and break-even points may shift with different hardware, sampling configurations and serving stacks.Their central recommendation is to measure acceptance on a representative sample of the intended workload rather than import a performance number from an unrelated benchmark.INCECT 2026 was technically co-sponsored by the IEEE Bangalore Section and IEEE North Karnataka Subsection.
