(2/2) The book covers the following topics:
β Mathematical foundations of machine learning and NLP
β Data preprocessing techniques for text data
β Machine learning applications for NLP and text classification
β Deep learning methods for NLP and text applications
β Theory and design of Large Language Models
β Applications of LLM models
β LLM applications with Langchain
The book is for folks who are interested in getting started with NLP and those who wish to delve into LLM applications.
Before I head off on a trip to various parts of not-Barcelona, I thought Iβd share a somewhat provocative paper by David Hogg and Soledad Villar. In my capacity as journal editor over the past few years Iβve noticed that there has been a phenomenal increase in astrophysics papers discussing applications of various forms of Machine Leaning (ML). This paper looks into issues around the use of ML not just in astrophysics but elsewhere in the natural sciences.
The abstract reads:
Machine learning (ML) methods are having a huge impact across all of the sciences. However, ML has a strong ontology β in which only the data exist β and a strong epistemology β in which a model is considered good if it performs well on held-out training data. These philosophies are in strong conflict with both standard practices and key philosophies in the natural sciences. Here, we identify some locations for ML in the natural sciences at which the ontology and epistemology are valuable. For example, when an expressive machine learning model is used in a causal inference to represent the effects of confounders, such as foregrounds, backgrounds, or instrument calibration parameters, the model capacity and loose philosophy of ML can make the results more trustworthy. We also show that there are contexts in which the introduction of ML introduces strong, unwanted statistical biases. For one, when ML models are used to emulate physical (or first-principles) simulations, they introduce strong confirmation biases. For another, when expressive regressions are used to label datasets, those labels cannot be used in downstream joint or ensemble analyses without taking on uncontrolled biases. The question in the title is being asked of all of the natural sciences; that is, we are calling on the scientific communities to take a step back and consider the role and value of ML in their fields; the (partial) answers we give here come from the particular perspective of physics
arXiv:2405.18095
P.S. The answer to the question posed in the title is probably βyesβ.
@metin That's interesting because in my circle (tech-savvy nerds and researchers) a lot of people use and recommend the use of ChatGPT. For example, the tutor of a scientific containerization course I attended last week used ChatGPT extensively to solve some very specific problems. Of course, you could get the same results using search engines, but an AI is much faster in these cases and can at least point you in the right direction.
@daniel Yes, I think it's a matter of time before AI will be widely used. Personally, I barely use ChatGPT, because I don't trust the output yet, due to the hallucinations. I'm waiting until that has been solved. But I know that it's already usable for exact purposes like coding.
For example, if we train a model to compute a simple, linear feature and a hard, highly non-linear one, the easy feature is naturally learned first, but both are generalized perfectly by the end of training. However, the easy feature dominates the representations! 3/9
This paper is really just us finally following up on a weird finding about RSA (figure on the here) from a paper Katherine Hermann & I had at NeurIPS back in the dark ages (2020): https://x.com/khermann_/status/1323353860283326464
Thanks to my coauthors @scychan_brains & Katherine! 9/9
Fine Tuning LLM Models β Generative AI Course ππΌ
FreeCodeCamp released today a new course for fine tuning LLM models. The course, by Krish Naik, focuses on different tuning methods such as QLORA, LORA, and Quantization using different models such as Llama2, Gradient, and Google Gemma model.
βThe Protein Universe Atlas is a groundbreaking resource for exploring the diversity of proteins. Its user-friendly web interface empowers researchers, biocurators and, students in navigating the βdark matterβ to explore proteins of unknown function.β
π₯ Thatβs what the committee said about this work, one of the #SIBRemarkableOutputs 2023 π
Soβ¦ Big Tech is allowed to blatantly steal the work, styles and therewith the job opportunities of thousands of artists and writers without being reprimanded, but it takes similarity to the voice of a famous actor to spark public outrage about AI. π€
The MLX is Apple's framework for machine learning applications on Apple silicon. The MLX examples repository provides a set of examples for using the MLX framework. This includes examples of:
β Text models such as transformer, Llama, Mistral, and Phi-2 models
β Image models such as Stable Diffusion
β Audio and speech recognition with OpenAI's Whisper
β Support for some Hugging Face models
@ramikrispin@BenjaminHan How do this and corenet (https://github.com/apple/corenet) fit together? The corenet repo has examples for inference with MLX for models trained with corenet; is that it, does MLX not have, e.g., activation and loss fns, optimizers, etc.?
@Lobrien@BenjaminHan The corenet is deep learning application where the MLX is array framework for high performance on Apple silicon. This mean that if you are using mac with M1-3 CPU it should perform better when using MLX on the backend (did not test it myself)
(1/2) MIT Introduction to Deep Learning πππ
MIT launched the 2024 edition of the Introduction to Deep Learning course by Prof. Alexander Amini and Prof.Ava Amini. The course started at the end of April and will run until June. The course lectures are published weekly. The course syllabus keeps changing from year to year, reflecting the rapid changes in this field.
(2/2) The course covers the following topics:
β Deep learning foundation
β Computer vision
β Deep generative modeling
β Reinforcement learning
β Robot learning
β Text to image
Deep Generative Models - New Stanford Course πππΌ
Stanford University released a new course last week focusing on Deep Generative Models. The course, by Prof. Stefano Ermon, focuses on the models beyond GenAI models.