Big Data Science

👍🏻Deep Learning helps you to be healthy!
For years, physicians have relied on visual inspection to identify suspicious pigmented lesions (SPLs), which can be an indication of skin cancer. Early-stage identification of SPLs can improve melanoma prognosis and significantly reduce treatment cost. But it is not easy quickly find and prioritize SPLs due to the high volume of pigmented lesions. Researchers from MIT have devised a new AI pipeline, using deep convolutional neural networks (DCNNs) and applying them to analyzing SPLs through the wide-field photography common in smartphones.
A wide-field image, acquired with a smartphone camera, shows large skin sections from a patient. An automated system detects, extracts, and analyzes all pigmented skin lesions observable in the wide-field image. A pre-trained DCNN ML-models determines the suspiciousness of individual pigmented lesions and marks them: further inspection as yellow, referral to dermatologist as red. Extracted features are used to further assess pigmented lesions and to display results in a heatmap format.
DCNNs are deep learning algorithms are used to classify images to then cluster them for performing a photo search.
https://news.mit.edu/2021/artificial-intelligence-tool-can-help-detect-melanoma-0402

😂If you do not want to study grammar and history, use ML to pass the exams! GPT-3 has done it with U.S. History, Research Methods, Creative Writing, and Law. In 3-20 minutes, NN was able to mimic human writing in areas of grammar, syntax, and word frequency and get the same feedback as the human writers
https://www.zdnet.com/article/ai-can-write-a-passing-college-paper-in-20-minutes/

🥺Looking for interesting reading? Take the TOP-21 books about Data Science, Engineering and Statistics – must to be read in 2021 https://towardsdatascience.com/21-data-science-books-you-should-read-in-2021-db625e97feb6
And short-list of 5 items https://medium.com/curious/5-books-every-data-scientist-should-read-in-2021-206609d8593b

🤓How to assess the potential effectiveness of medical drugs: new method DeepBAR form MIT researchers to calculate the binding affinities between drug candidates and their targets. It is based on GAN-models for analyzing molecular structures as images
https://news.mit.edu/2021/drug-discovery-binding-affinity-0315

👀 Why modern AI for Computer Vision should have Multimodal Neurons and how this Faceted Feature Visualization rises the accuracy of predictions and classifications. New paper from
OpenAI researchers https://distill.pub/2021/multimodal-neurons/

💥Meet the CLIP (Contrastive Language – Image Pre-training) - new Neural Net from OpenAI: it can be instructed in natural language to perform a great variety of classification benchmarks, without directly optimizing for the benchmark’s performance, similar to the “zero-shot” capabilities of GPT-2 and GPT-3. CLIP is based on zero-shot transfer, natural language supervision, and multimodal learning to recognize a wide variety of visual concepts in images and associate them with their names. Read more where you can use this unique ML-model https://openai.com/blog/clip/

😜Not only Deep Learning: new approach to build AI systems working as human brain - sparse coding principle to supply series of local functions in synaptic learning rules and reduce number of adjusting data in NN-model. The startup Nara Logics from MIT alumnus is trying to increase effectiveness of AI by mimicking the brain structure and function at the circuit level.
https://news.mit.edu/2021/nara-logics-ai-0312

🤓Deep fake is not too simple: interview with Belgium VFX specialist Chris Ume, creator of viral video about fake Tom Cruise. Why only ML-algorithm is not enough to get high quality result and you need thorough tune video effects manually
https://www.theverge.com/2021/3/5/22314980/tom-cruise-deepfake-tiktok-videos-ai-impersonator-chris-ume-miles-fisher

About tensor holography to create real time 3D-holograms for virtual reality, 3D printing and medical visualization that could be run on your smartphone. Meet new AI-method from MIT researchers https://news.mit.edu/2021/3d-holograms-vr-0310

Deep into NGBoost and probabilistic regression: what is probabilistic supervised learning and how to deal with prediction intervals. About correct interpretation of this ML-algorithm
https://towardsdatascience.com/interpreting-the-probabilistic-predictions-from-ngboost-868d6f3770b2

ML to optimize microchip's architecture: Apollo project from Google to search right parameters of chip for certain Neural Net and meet the high speed of computations
https://www.zdnet.com/article/googles-deep-learning-finds-a-critical-path-in-ai-chips

😂For those who skipped everything in 2020: Top 15 Machine Learning & AI Research Papers – from YOLO 4 to TensorFlow Quantum https://rubikscode.net/2020/12/21/2020s-top-15-machine-learning-ai-research-papers/

🌷Not only LightGBM and XGBoost: meet new probabilistic prediction algorithm - Natural Gradient Boosting (NGBoost). Released in 2019, NGBoost uses the Natural Gradient to address technical challenges that makes generic probabilistic prediction hard with existing gradient boosting methods. This algorithm consists of three abstract modular components: base learner, parametric probability distribution, and scoring rule. All three components are treated as hyperparameters chosen in advance before training. NGBoost makes it easier to do probabilistic regression with flexible tree-based models. Further, it has been possible to do probabilistic classification for quite some time since most classifiers are actually probabilistic classifiers in that they return probabilities over each class. For instance, logistic regression returns class probabilities as output. In this light, NGBoost doesn’t add much new but experiments on several regression datasets proved that this ML-algorithm provides competitive predictive performance of both uncertainty estimates and traditional metrics. On other hand its computing time is quite longer than other two algorithms and there’s no some useful options, e.g. early stopping, showing the intermediate results, the flexibility of choosing the base learner, setting a random state seed, dealing only with decision tree and Ridge regression,and so on. But this modular ML-algorithm for probabilistic prediction is quite competitive against other popular boosting methods. See more
http://www.51anomaly.org/pdf/NGBOOST.pdf
https://medium.com/@ODSC/using-the-ngboost-algorithm-8d337b753c58
https://towardsdatascience.com/ngboost-explained-comparison-to-lightgbm-and-xgboost-fda510903e53
https://www.groundai.com/project/ngboost-natural-gradient-boosting-for-probabilistic-prediction/1

💦Transparent interpretation of results and permanent learning in production with non-stop adaptation of neural network to new conditions and data
Liquid NN from MIT for decision making in autonomous driving and medical diagnosis based on nervous system of microscopic nematode with 302 neurons and principles of time series data ananlytics. This ML-model edged out other state-of-the-art time series algorithms by a few percentage points in accurately predicting future values in datasets, ranging from atmospheric chemistry to traffic patterns. Just changing the representation of a neuron with the differential equations, you can deal with small number of highly expressive neurons and peer into the “black box” of the network’s decision making and diagnose why the network made a certain characterization.
https://news.mit.edu/2021/machine-learning-adapts-0128

🌞New look to Elasticsearch: how to use it as Time Series Database and tune performance of queries for interactive data analytics - case of ThousandEyes company https://medium.com/thousandeyes-engineering/what-we-learned-using-elasticsearch-as-a-time-series-database-bdbde38cdb64

How to streamline the implementation of reasoning systems with ReAgent from Facebook.
ReAgent is the end-to-end platform applied Reinforcement Learning designed for large-scale, distributed recommendation/optimization tasks where we don’t have access to a simulator. The main purpose of this framework is to make the development & experimentation of deep reinforcement algorithms fast. ReAgent is built on Python. It uses PyTorch framework for data modelling. ReAgent holds different algorithms for data preprocessing, feature engineering, model training & evaluation and lastly for optimized serving. It is capable of handling Large-dimension datasets, provides optimized algorithms for data preprocessing, training, and gives a highly efficient production environment for model serving. https://analyticsindiamag.com/hands-on-to-reagent-end-to-end-platform-for-applied-reinforcement-learning/

Typical use cases for the most popular ML-algorithms with BigQuery - from regression to time series analysis
https://medium.com/cloudzone/try-62d6aeb4a5e1

Why Data Engineer is the best friend of Data Scientist and Data Analyst - the shifting role in 2021: tasks, responsibilities, salaries and perspectives
https://palakdatascientist.medium.com/data-engineers-of-2021-the-shifting-role-35d13c9106f

🎨 Colorization Transformer by Google

💻 Github: https://github.com/google-research/google-research/tree/master/coltran

📝 Paper: https://arxiv.org/abs/2102.04432v1

🌐 Pre-trained model: https://console.cloud.google.com/storage/browser/gresearch/coltran;tab=objects?pli=1&prefix=&forceOnObjectsSortingFiltering=false

@ai_machinelearning_big_data

👀How to find medical image from billions of templates through Spark SQL: scalable library to efficiently read DICOM-files in Dataframe https://bd-practice.medium.com/dicom-read-library-apache-spark-third-party-contribution-e6cb269e5c3c