Does RAG Even Scale? EyeLevel vs LangChain

OVERFIT: AI, Machine Learning, and Deep Learning Made Simple

Player FM - Internet Radio Done Right

เพิ่มแล้วเมื่อ twenty-nine สัปดาห์ที่ผ่านมา
Looks like the publisher may have taken this series offline or changed its URL. Please contact support if you believe it should be working, the feed URL is invalid, or you have any other concerns about it.

เนื้อหาจัดทำโดย Brian Carter เนื้อหาพอดแคสต์ทั้งหมด รวมถึงตอน กราฟิก และคำอธิบายพอดแคสต์ได้รับการอัปโหลดและจัดหาให้โดยตรงจาก Brian Carter หรือพันธมิตรแพลตฟอร์มพอดแคสต์ของพวกเขา หากคุณเชื่อว่ามีบุคคลอื่นใช้งานที่มีลิขสิทธิ์ของคุณโดยไม่ได้รับอนุญาต คุณสามารถปฏิบัติตามขั้นตอนที่แสดงไว้ที่นี่ https://th.player.fm/legal

All About Change

1
Professional football player Jonathan Jones: Mentorship and Making an Impact in Your Community 22:49

11 วันที่แล้ว22:49

ลิสต์เล่นในภายหลัง

ลิสต์

ถูกใจ

ที่ถูกใจแล้ว

22:49

Jonathan Jones is an NFL cornerback for the Washington Commanders who rose from the undrafted ranks to become two-time Super Bowl champion with the New England Patriots, a businessman, philanthropist, and licensed pilot. In 2019, Jonathan founded the Jonathan Jones Next Step Foundation in 2019, a platform dedicated to empowering youth through education, professional development, and mentorship. The foundation works to alleviate food insecurity, promote women in stem and sports, and to promote professional development in the communities where he lives. Jay and Jonathan talk about investing in the communities they live in, acknowledging the people who helped you become the person you are, and paying that same investment forward to the next generation. Episode Chapters 0:00 intro 1:24 Building local connections 4:25 Jonathan’s mentors and mentees 10:54 Jonathan’s pride in his mentees’ successes 13:04 how Jonathan chooses his causes 14:08 Jonathan’s support for girls and young women 17:19: Jonathan’s passion for flying 19:40 The Next Step Foundation 20:29 Goodbye For video episodes, watch on www.youtube.com/@therudermanfamilyfoundation Stay in touch: X: @JayRuderman | @RudermanFdn LinkedIn: Jay Ruderman | Ruderman Family Foundation Instagram: All About Change Podcast | Ruderman Family Foundation To learn more about the podcast, visit https://allaboutchangepodcast.com/ Looking for more insights into the world of activism? Be sure to check out Jay’s brand new book, Find Your Fight , in which Jay teaches the next generation of activists and advocates how to step up and bring about lasting change. You can find Find Your Fight wherever you buy your books, and you can learn more about it at www.jayruderman.com .…

ประมาณหนึ่งปีที่แล้ว 7:55

MP3•หน้าโฮมของตอน

Fetch error

Hmmm there seems to be a problem fetching this series right now. Last successful fetch was on November 09, 2024 13:09 (6M ago)

What now? This series will be checked again in the next day. If you believe it should be working, please verify the publisher's feed link below is valid and includes actual episode links. You can contact support to request the feed be immediately fetched.

A research team from EyeLevel.ai has found that vector databases, which are commonly used in RAG (Retrieval-Augmented Generation) systems, have a scaling problem. Their research shows that the accuracy of vector similarity search degrades significantly as the number of pages in the database increases, leading to a substantial performance hit. This problem can be attributed to the way modern encoders organize information in high-dimensional vector spaces. In contrast, EyeLevel's RAG platform, which does not rely on vectors, demonstrates superior performance at scale, losing only 2% accuracy with 100,000 pages. The team's findings highlight the need for developers to be aware of these challenges when scaling RAG applications in production.

71 ตอน

Does RAG Even Scale? EyeLevel vs LangChain

OVERFIT: AI, Machine Learning, and Deep Learning Made Simple

published ประมาณหนึ่งปีที่แล้ว

แบ่งปัน

MP3•หน้าโฮมของตอน

Fetch error

Hmmm there seems to be a problem fetching this series right now. Last successful fetch was on November 09, 2024 13:09 (6M ago)

71 ตอน

ทุกตอน

1
Does the DIFF Transformer make a Diff? 8:03

24 weeksที่แล้ว8:03

8:03

Introducing a novel transformer architecture, Differential Transformer, designed to improve the performance of large language models. The key innovation lies in its differential attention mechanism, which calculates attention scores as the difference between two separate softmax attention maps. This subtraction effectively cancels out irrelevant context (attention noise), enabling the model to focus on crucial information. The authors demonstrate that Differential Transformer outperforms traditional transformers in various tasks, including long-context modeling, key information retrieval, and hallucination mitigation. Furthermore, Differential Transformer exhibits greater robustness to order permutations in in-context learning and reduces activation outliers, paving the way for more efficient quantization. These advantages position Differential Transformer as a promising foundation architecture for future large language model development. Read the research here: https://arxiv.org/pdf/2410.05258…

1
Automating Scientific Discovery: ScienceAgentBench 9:49

24 weeksที่แล้ว9:49

9:49

Introducing, ScienceAgentBench, a new benchmark for evaluating language agents designed to automate scientific discovery. The benchmark comprises 102 tasks extracted from 44 peer-reviewed publications across four disciplines, encompassing essential tasks in a data-driven scientific workflow such as model development, data analysis, and visualization. To ensure scientific authenticity and real-world relevance, the tasks were validated by nine subject matter experts. The paper presents an array of evaluation metrics for assessing program execution, results, and costs, including a rubric-based approach for fine-grained evaluation. Through comprehensive experiments on five LLMs and three frameworks, the study found that the best-performing agent, Claude-3.5-Sonnet with self-debug, could only solve 34.3% of the tasks using expert-provided knowledge. These findings highlight the limitations of current language agents in fully automating scientific discovery, emphasizing the need for more rigorous assessment and future research on improving their capabilities for data processing and utilizing expert knowledge. Read the paper: https://arxiv.org/pdf/2410.05080…

1
Prune This! PyTorch and Efficient AI 8:04

24 weeksที่แล้ว8:04

8:04

Both sources explain neural network pruning techniques in PyTorch. The first source, "How to Prune Neural Networks with PyTorch," provides a general overview of the pruning concept and its various methods, along with practical examples of how to implement different pruning techniques using PyTorch's built-in functions. The second source, "Pruning Tutorial," focuses on a more in-depth explanation of pruning functionalities within PyTorch, demonstrating how to prune individual modules, apply iterative pruning, serialize pruned models, and even extend PyTorch with custom pruning methods. Read this: https://towardsdatascience.com/how-to-prune-neural-networks-with-pytorch-ebef60316b91 And the PyTorch tutorial: https://pytorch.org/tutorials/intermediate/pruning_tutorial.html…

1
AlexWho? Going Deeper with Deep CNNs 11:50

24 weeksที่แล้ว11:50

11:50

The source is a chapter from the book "Dive into Deep Learning" that explores the historical development of deep convolutional neural networks (CNNs), focusing on the foundational AlexNet architecture. The authors explain the challenges faced in training CNNs before the advent of AlexNet, including limited computing power, small datasets, and lack of crucial training techniques. They discuss how AlexNet overcame these obstacles by leveraging powerful GPUs, large-scale datasets like ImageNet, and innovative training strategies. The chapter also delves into the architecture of AlexNet, highlighting its similarities to LeNet, and comparing its advantages in terms of depth, activation function, and model complexity control. Finally, the authors emphasize the importance of AlexNet as a crucial step towards the development of the deep networks used today, showcasing its impact on the field of computer vision and deep learning. Read more: https://d2l.ai/chapter_convolutional-modern/alexnet.html…

1
Predicting the Future from the Past: Sequential RNN Stuff 9:47

24 weeksที่แล้ว9:47

9:47

This text is an excerpt from the "Dive into Deep Learning" book, specifically focusing on the processing of sequential data. The authors introduce the challenges of working with data that occurs in a specific order, like time series or text, and how these sequences cannot be treated as independent observations. They delve into autoregressive models, where future values are predicted based on past values, and highlight the common problem of error accumulation when predicting further into the future. The text discusses the concept of Markov models, where only a limited history is needed to predict future events, as well as the importance of understanding the causal structure of the data. The excerpt then provides a practical example of using linear regression for autoregressive modeling on synthetic time series data and demonstrates the limitations of simple models for long-term prediction. Read more: https://d2l.ai/chapter_recurrent-neural-networks/sequence.html…

1
Google's Secrets to Getting People to Adopt A.I. 8:39

25 weeksที่แล้ว8:39

8:39

This excerpt from "Mental Models," a chapter in the "People + AI Guidebook," focuses on the importance of understanding and managing user mental models when designing AI-powered products. The authors discuss how to set expectations for adaptation, onboard users in stages, plan for co-learning, and account for user expectations of human-like interaction. By carefully considering these factors, product designers can ensure that users form accurate mental models and have a positive experience with AI-powered products. Read more here: https://pair.withgoogle.com/chapter/mental-models/…

1
LLM Tokenizers, from HFs LNP Course 12:23

25 weeksที่แล้ว12:23

12:23

This excerpt from Hugging Face's NLP course provides a comprehensive overview of tokenization techniques used in natural language processing. Tokenizers are essential tools for transforming raw text into numerical data that machine learning models can understand. The text explores various tokenization methods, including word-based, character-based, and subword tokenization, highlighting their advantages and disadvantages. It then focuses on the encoding process, where text is first split into tokens and then converted to input IDs. Finally, the text demonstrates how to decode input IDs back into human-readable text. Read more: https://huggingface.co/learn/nlp-course/en/chapter2/4…

1
PyTorch vs Tensorflow: Who Wins in CNN? 11:49

25 weeksที่แล้ว11:49

11:49

This research paper examines the efficiency of two popular deep learning libraries, TensorFlow and PyTorch, in developing convolutional neural networks. The authors aim to determine if the choice of library impacts the overall performance of the system during training and design. They evaluate both libraries using six criteria: user-friendliness, available documentation, ease of integration, overall training time, overall accuracy, and execution time during evaluation. The paper proposes a novel methodology for comparing these libraries by eliminating external factors that could influence the comparison and focusing solely on the six chosen criteria. The study finds that while both libraries offer similar capabilities, PyTorch is better suited for tasks that prioritize speed and ease of use, while TensorFlow excels in tasks demanding accuracy and flexibility. The authors conclude that the choice of library has a significant impact on both design and performance and that the presented criteria can assist users in selecting the most appropriate library for their specific needs. Read more: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9699128/pdf/sensors-22-08872.pdf…

1
Google's 43 Rules for Machine Learning 9:28

25 weeksที่แล้ว9:28

9:28

This document provides a comprehensive set of rules for building and deploying machine learning systems, focusing on best practices gleaned from Google’s extensive experience. The document is divided into sections that cover the key stages of the machine learning process, including launching a product without ML, designing and implementing metrics, creating a first pipeline, feature engineering, human analysis, and addressing the challenges of training-serving skew. The rules cover a wide range of topics, from choosing the right objective function to detecting silent failures, and from creating human-understandable features to avoiding feedback loops. The document also offers guidance for navigating the transition from simple to more complex models as a system matures and performance plateaus. Go deeper here: https://developers.google.com/machine-learning/guides/rules-of-ml…

1
Do we Need the Mamba Mindset when LLMs Fail? MoE Mamba and SSMs 11:57

25 weeksที่แล้ว11:57

11:57

The research paper "MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts" explores a novel approach to language modeling by combining State Space Models (SSMs), which offer linear-time inference and strong performance in long-context tasks, with Mixture of Experts (MoE), a technique that scales model parameters while minimizing computational demands. The authors introduce MoE-Mamba, a model that interleaves Mamba, a recent SSM-based model, with MoE layers, resulting in significant performance gains and training efficiency. They demonstrate that MoE-Mamba outperforms both Mamba and standard Transformer-MoE architectures. The paper also explores different design choices for integrating MoE within Mamba, showcasing promising directions for future research in scaling language models beyond tens of billions of parameters. Read it: https://arxiv.org/abs/2401.04081…

1
Agentic Retrieval Augmented Generation (RAG) systems 7:39

25 weeksที่แล้ว7:39

7:39

We discuss how to build Agentic Retrieval Augmented Generation (RAG) systems, which use AI agents to retrieve information from various sources to answer user queries. The author details the challenges he faced when building an Agentic RAG system to answer customer support questions, and provides insights into techniques like prompt engineering and structured responses that helped him achieve better results. He also discusses best practices for building effective Agentic RAG systems and touches on the future of AI agents, including multi-agent systems and autonomous agents. Read more: https://arxiv.org/pdf/2408.08435…

1
Let's Get Activated! Why Non-Linear Activation Matters 7:15

25 weeksที่แล้ว7:15

7:15

Let's get RE(a)L, U! This research paper explores the impact of different activation functions, specifically ReLU and L-ReLU, on the performance of deep learning models. The authors investigate how the choice of activation function, along with factors like the number of parameters and the shape of the model architecture, influence model accuracy across various data domains (continuous, categorical with and without transfer learning). The study concludes that L-ReLU is more effective than ReLU when the number of parameters is relatively small, while ReLU generally performs better with larger models. The paper also highlights the importance of considering the specific data domain and the use of pre-trained models for transfer learning when selecting the most suitable activation function. Read more: https://github.com/christianversloot/machine-learning-articles/blob/main/why-nonlinear-activation-functions-improve-ml-performance-with-tensorflow-example.md…

1
Reviewing Stanford on Linear Regression and Gradient Descent 8:25

26 weeksที่แล้ว8:25

8:25

This lecture from Stanford University's CS229 course, "Machine Learning," focuses on the theory and practice of linear regression and gradient descent, two fundamental machine learning algorithms. The lecture begins by motivating linear regression as a simple supervised learning algorithm for regression problems where the goal is to predict a continuous output based on a set of input features. The lecture then introduces the cost function used in linear regression, which measures the squared error between the predicted output and the true output. Gradient descent, an iterative algorithm, is then explained as a method to find the parameters that minimize the cost function. Two variants of gradient descent, batch gradient descent and stochastic gradient descent, are discussed with their respective strengths and weaknesses. The lecture concludes with a derivation of the normal equations, an alternative approach to finding the optimal parameters in linear regression that involves solving a system of equations rather than iteratively updating parameters. Watch Andrew Ng teach it at Stanford: https://www.youtube.com/watch?v=4b4MUYve_U8&t=1086s&pp=ygUSdmFuaXNoaW5nIGdyYWRpZW50…

27 weeksที่แล้ว7:55

7:55

OVERFIT: AI, Machine Learning, and Deep Learning Made Simple

1
WHY Are Probability And Stats Foundational to ML and DL? 13:39

27 weeksที่แล้ว13:39

13:39

Probability and statistics are fundamental components of machine learning (ML) and deep learning (DL) because they provide the mathematical framework for understanding and analyzing data, which is crucial for making predictions and decisions. This excerpt from the "Dive into Deep Learning" documentation explains the essential concepts of probability and statistics, which are crucial for understanding machine learning. The text introduces fundamental ideas like sample space, events, probability functions, and random variables, highlighting the distinction between discrete and continuous variables. It then delves into the relationships between multiple random variables, emphasizing the importance of conditional probabilities, Bayes' Theorem, and independence. The excerpt also covers expectations and variances, illustrating how they can be used to measure the average value and the spread of data. Finally, it explores the concepts of aleatoric and epistemic uncertainty, providing a framework for understanding the limitations of machine learning models and the role of data in improving their accuracy. Read more: https://d2l.ai/chapter_preliminaries/probability.html…

OVERFIT: AI, Machine Learning, and Deep Learning Made Simple

1
Finally? Super-Accurate Weather Forecasts with AI 8:04

27 weeksที่แล้ว8:04

8:04

This research paper examines a new deep-learning approach to optimizing weather forecasts by adjusting initial conditions. The authors test their method on the 2021 Pacific Northwest heatwave, finding that small changes to initial conditions can significantly improve the accuracy of 10-day forecasts using both the GraphCast and Pangu-Weather deep-learning models. Notably, the improvements seem to stem primarily from correcting analysis errors, rather than model errors. The study suggests that this method could potentially increase the accuracy of weather forecasts beyond the current 10-day limit, potentially extending the limit to 23 days. Read more here: https://agupubs.onlinelibrary.wiley.com/doi/10.1029/2024GL110651…

OVERFIT: AI, Machine Learning, and Deep Learning Made Simple

1
Let's Descend: Why Calculus is Important in Deep Learning 11:00

28 weeksที่แล้ว11:00

11:00

An introduction to the fundamental concepts of calculus, explaining how they are essential for understanding deep learning. It begins by illustrating the concept of a limit using the calculation of a circle's area, before introducing the concept of a derivative, which describes a function's rate of change. It then extends these concepts to multivariate functions, discussing partial derivatives and gradients, which are crucial for optimizing models in deep learning. The chain rule, a powerful tool for calculating gradients in complex function compositions, is also explained in detail. The text concludes by highlighting the significance of automatic gradient computation, emphasizing its role in optimizing deep learning models and paving the way for the backpropagation algorithm, which will be elaborated on in later chapters. Read more: https://d2l.ai/chapter_preliminaries/calculus.html…

OVERFIT: AI, Machine Learning, and Deep Learning Made Simple

1
OMG OpenAI's o1 is a LogicQuake! The AI Reasoning Layer, Sequoia Capital 6:42

28 weeksที่แล้ว6:42

6:42

The source, "Generative AI's Act o1: The Reasoning Era Begins | Sequoia Capital," discusses the evolution of AI models from simply mimicking patterns to engaging in more deliberate reasoning. The authors argue that the next frontier in AI is the development of "System 2" thinking, where models can reason through complex problems and make decisions based on logic rather than simply relying on pre-trained data. The article highlights OpenAI's o1 model as an example of this evolution, as it incorporates inference-time compute, allowing the model to "stop and think" before responding. The authors then explore the implications of this shift for AI applications, arguing that a new generation of "agentic applications" is emerging that can perform tasks previously done by humans, such as customer support or legal analysis. The article concludes by examining the investment opportunities presented by this new era of AI, suggesting that the most promising areas for venture capital are in the development of cognitive architectures and "agentic applications" that can address the complexities of the real world. Read the article here: https://www.sequoiacap.com/article/generative-ais-act-o1/…

ขอต้อนรับสู่ Player FM!

Player FM กำลังหาเว็บ

เปิดฟังกว่า 500+ หัวข้อ

Fourth Wing (The Empyrean Book 1)

Nippies Nipple Cover - Sticky Adhesive Silicone Nipple Pasties - Reusable Pasty Nipple Covers for Women with Travel Box (Small (Fits A-C Cups), Original, Crème)

Amazon Basics Hypoallergenic 100% Cotton Rounds, 100 Count

พอดคาสต์ที่ควรค่าแก่การฟัง

OVERFIT: AI, Machine Learning, and Deep Learning Made Simple « » Does RAG Even Scale? EyeLevel vs LangChain

Fetch error

Does RAG Even Scale? EyeLevel vs LangChain

Fetch error

พอดคาสต์ที่ควรค่าแก่การฟัง

ขอต้อนรับสู่ Player FM!

Mighty Patch™ Original patch from Hero Cosmetics - Hydrocolloid Acne Pimple Patch for Covering Zits and Blemishes in Face and Skin, Vegan-friendly and Not Tested on Animals (36 Count)

Onyx Storm (The Empyrean Book 3)

Grad Cap Remix - The ORIGINAL Graduation Cap Headband - Keep Your Natural Hair on Grad Day and Secure Your Cap

Niidor Adhesive Bra Strapless Sticky Invisible Push up Silicone Bra for Backless Dress with Nipple Covers Nude(B Cup)

คู่มืออ้างอิงด่วน

OVERFIT: AI, Machine Learning, and Deep Learning Made Simple « »
Does RAG Even Scale? EyeLevel vs LangChain