https://blog.takanabe.tokyo/en/ Recent content on Payforward Hugo -- gohugo.io en-us Mon, 18 Sep 2023 12:00:00 +0900 https://blog.takanabe.tokyo/en/2023/09/ee194398-aff2-4065-bf2a-1acbe7b081d5/ Mon, 18 Sep 2023 12:00:00 +0900 https://blog.takanabe.tokyo/en/2023/09/ee194398-aff2-4065-bf2a-1acbe7b081d5/ Motivation In my article, Evaluating ragas: A framework for RAG pipelines, I explored the workings of ragas and the metrics it emphasizes. This piece offers a deeper examination of those metrics. Performance Metrics ragas evaluates RAG pipeline’s performance using the following dimensions: Faithfulness: Ensures the generated answer’s consistency with the given context. Penalties are given for claims not deduced from the context. Calculated using the answer and retrieved context. https://blog.takanabe.tokyo/en/2023/09/6e5629d2-5dea-4203-b6c4-8a0ac352b032/ Sun, 17 Sep 2023 12:00:00 +0900 https://blog.takanabe.tokyo/en/2023/09/6e5629d2-5dea-4203-b6c4-8a0ac352b032/ TL;DR ragas is a framework designed to assess the performance of Retrieval Augmented Generation (RAG) pipelines, a type of LLM application that uses external data to enhance its context. While there are tools to build RAG pipelines, evaluating their performance can be challenging. ragas offers tools rooted in recent research to evaluate LLM-generated text, providing insights into RAG pipeline performance. It can also be incorporated into CI/CD for ongoing performance checks. https://blog.takanabe.tokyo/en/2023/09/834d7288-8621-4e16-87a3-7fdc955d55bc/ Sat, 16 Sep 2023 15:00:00 +0900 https://blog.takanabe.tokyo/en/2023/09/834d7288-8621-4e16-87a3-7fdc955d55bc/ TL;DR Databricks offers a detailed exploration of the evaluation techniques specific to LLM applications in RAG. The emphasis is on the importance of robust metrics and the nuanced intricacies of the evaluation process. Motivation In the ever-evolving world of Natural Language Processing, understanding how to evaluate Language Model applications is paramount. This article, inspired by Databricks’ “Best Practices for LLM Evaluation of RAG Applications”, delves deep into the metrics, methodologies, and challenges of evaluating LLM applications using Retrieval-Augmented Generation (RAG). https://blog.takanabe.tokyo/en/2022/06/40-f87-d38-0-e07-4-c50-a884-6-d5-a0-c054318/ Sun, 19 Jun 2022 08:00:00 +0900 https://blog.takanabe.tokyo/en/2022/06/40-f87-d38-0-e07-4-c50-a884-6-d5-a0-c054318/ TL;DR As for collection usages in collaborative projects , I prefer to use Guava instead of Java API for consistency, simplicity, and safety. Use immutable collections if we don’t need mutable ones to take advantages of performance and get rid of mis-operations around collections. JDK’s Collections.unmodifiableXXX is an error-prone and need to understand the behavior deeply (not good for team-wise projects). Motivation Still not productive in Java due to my unfamiliarity. https://blog.takanabe.tokyo/en/2022/02/4a30422a-d158-4c43-a808-c3e389301831/ Tue, 15 Feb 2022 08:00:00 +0900 https://blog.takanabe.tokyo/en/2022/02/4a30422a-d158-4c43-a808-c3e389301831/ Motivation Understand how to write an enum type in Java. Basic definitions and instance methods public class Main { public static void main(String[] args) { try { System.out.printf("LogLevel.valueOf(): %s\n", LogLevel.valueOf("LOW")); System.out.printf("LogLevel.valueOf().getClass(): %s\n", LogLevel.valueOf("LOW").getClass()); System.out.printf("LogLevel.valueOf(): %s\n", LogLevel.valueOf("NONE")); } catch (Exception e){ System.out.println(e); } System.out.println(); // Testing enum's constructor for (LogLevel logLevel: LogLevel.values()) { System.out.println("--------------------START-----------------------"); System.out.printf("# Log level: %s\n", logLevel); System.out.printf("instance of LogLevel?: %b\n",logLevel instanceof LogLevel); System.out.println(); // Testing getter method System. https://blog.takanabe.tokyo/en/2022/02/ce01dc9b-1d3f-4afb-b9a4-161acd5e3e0e/ Mon, 14 Feb 2022 09:30:00 +0900 https://blog.takanabe.tokyo/en/2022/02/ce01dc9b-1d3f-4afb-b9a4-161acd5e3e0e/ TL;DR JEP 330 allows us to run casual (e.g. scripting purpose) Java code without javac Traditional Java code execution Let’s say we have the following Main.java and Word.java. // Main.java public class Main { public static void main(String[] args) { for (Word str : Word.values()) { System.out.println(str.getWord()); } } } // Word.java public enum Word { A("apple"), B("banana"), C("cat"), D("dog") ; private final String str; Word(String str) { this. https://blog.takanabe.tokyo/en/2022/01/312b7f26-be12-4ff8-91fd-aaca65f729f6/ Sat, 15 Jan 2022 06:00:00 +0900 https://blog.takanabe.tokyo/en/2022/01/312b7f26-be12-4ff8-91fd-aaca65f729f6/ TL;DR I had two-months sabbatical since Oct, 2021 I resigned Cookpad Inc. and joined Launchable, Inc. Journey at Cookpad is closed As described in my Japanese article, after closing my memorable journey at Cookpad Inc., I joined Launchable, Inc. I had many interesting technical challenges and could enthusiastically tackle them every day. Because I spent almost six years at the multi-national company, I have many friends around the globe, which is the most valuable for me more than anything. https://blog.takanabe.tokyo/en/2022/01/ac0d08e7-90a3-4242-97df-4b5b82b2e7ae/ Thu, 13 Jan 2022 06:30:00 +0900 https://blog.takanabe.tokyo/en/2022/01/ac0d08e7-90a3-4242-97df-4b5b82b2e7ae/ The following comparison tables are useful to understand differences between CicleCI and GitHub Actions services as of Jan 13, 2022. I created this table to ellaborate on the differences between two most used CI SaaS for small/middle size organizations. TL;DR GitHub Actions and CircleCI have almost same features If you start using CI SaaS today, use GitHub actions If you want to support a monorepo, use GitHub actions If you need huge concurrency capacity but don’t want to manage self-hosted runners, use CircleCI Feature comparison Feature Expectation Circle CI GitHub Actions Note Manual CI execution We can trigger CI jobs by manually outside of CI/CD pipelines. https://blog.takanabe.tokyo/en/2020/05/error-handling-in-golang-1.13/ Fri, 08 May 2020 13:00:00 +0900 https://blog.takanabe.tokyo/en/2020/05/error-handling-in-golang-1.13/ TL;DR errors standard library package in golang 1.13 cannot be alternatives of pkg/errors and xerrors. Use xerrors for error handling for new golang projects. No need to switch from pkg/errors to xerrors. Motivation golang is a very simple language and that’s the core philosophy of its design. Along with the design, standard libraries are also simple and don’t do too many things. However, the features of error standard library is not enough for developments. https://blog.takanabe.tokyo/en/2020/04/can-github-cli-be-alternative-of-hub/ Sat, 25 Apr 2020 07:00:00 +0900 https://blog.takanabe.tokyo/en/2020/04/can-github-cli-be-alternative-of-hub/ TL;DR The GitHub CLI replaces the hub if you’re using only github.com GitHub Enterprise is not supported now (as of 2020/4/24), so it’s not a replacement for hub for my use. The GitHub CLI now supports GHE !! See GA for more info. Motivation Since GitHub CLI can do more things, I investigated whether I can migrate from hub, which is usually used as a CLI client of GitHub. https://blog.takanabe.tokyo/en/2020/04/remove-squash-merged-local-git-branches/ Sun, 19 Apr 2020 22:45:59 +0900 https://blog.takanabe.tokyo/en/2020/04/remove-squash-merged-local-git-branches/ Motivation The default merge method on GitHub Pull Requests creates a merge commit. But GitHub also provides us several merge options and “Squash and merge” option is one of them. Because a squash-merged commit is not the same as the commits in a local branch, git cannot list the squash-merged local branch by git branch --merged. Therefore, we cannot clean up outdated local branches with the following commands. $ git fetch --prune $ git branch --merged | egrep -v "(^\*|master|ANY_BRANCH_YOU_WANT_TO_EXCLUDE)" | xargs -I % git branch -d % These commands can remove obsolete remote branches on GitHub and local branches when we use default merge commits. https://blog.takanabe.tokyo/en/2020/04/elasticsearch-basic-for-operators-part-1-terminology-in-elasticsearch-world/ Sun, 05 Apr 2020 23:00:59 +0900 https://blog.takanabe.tokyo/en/2020/04/elasticsearch-basic-for-operators-part-1-terminology-in-elasticsearch-world/ Motivation Elasticsearch is one of the candidates to provide search and auto-completion features to web and mobile applications. I’ve delivered Elasticsearch in production for many years in various services. However, I’m stupid enough to forget its technical jargon and essential concepts, configurations, and operations. It’s a waste of time to investigate the Elastic official documents again and again because the page I looked for was always the same. Thus, I will leave the important information to deliver Elasticsearch in production from my experiences. https://blog.takanabe.tokyo/en/2019/06/strategy-for-two-tier-on-call-rotation/ Tue, 25 Jun 2019 14:44:55 +0000 https://blog.takanabe.tokyo/en/2019/06/strategy-for-two-tier-on-call-rotation/ TL;DR I tried to elaborate on the pros and cons of single-, dual-, and triple-site on-call rotations. Single-site on-call rotation is straightforward and easy to deliver Dual-site on-call rotation brings huge advantages such as the promise of being able to sleep and have private time, as well as talent pool expansions, if we can break barriers to communication Triple-site on-call rotation is not feasible because of the significant cost for communication and delivery Motivation I have been working as a Site Reliability Engineer (SRE) for a while now. https://blog.takanabe.tokyo/en/2019/06/an-elegant-puzzle-systems-of-engineering-management-chapter-2/ Wed, 05 Jun 2019 01:06:51 +0000 https://blog.takanabe.tokyo/en/2019/06/an-elegant-puzzle-systems-of-engineering-management-chapter-2/ Motivation This is a book reading record of “An Elegant Puzzle: Systems of Engineering Management [Chapter 2]” An Elegant Puzzle: Systems of Engineering Management (English Edition) Author: Will Larson Manufacturer: Stripe Press Publish date: 2019-05-28 The main purpose of reading this book is gaining knowledge to lead teams in rapid growth organizations. It seems that the author had experienced those kinds of significant organization growth at Uber and Stripe (organization size changed from 200 to 2000) and I think the insights and experiences accumulated in the chaotic environment must be valuable for teams which will face the same kinds of growth and problems embraced by the growth. https://blog.takanabe.tokyo/en/2019/06/an-elegant-puzzle-systems-of-engineering-management-chapter-1/ Sun, 02 Jun 2019 15:37:47 +0000 https://blog.takanabe.tokyo/en/2019/06/an-elegant-puzzle-systems-of-engineering-management-chapter-1/ Motivation This is a book reading record of “An Elegant Puzzle: Systems of Engineering Management [Chapter 1]” An Elegant Puzzle: Systems of Engineering Management (English Edition) Author: Will Larson Manufacturer: Stripe Press Publish date: 2019-05-28 The main purpose of reading this book is gaining knowledge to lead teams in rapid growth organizations. It seems that the author had experienced those kinds of significant organization growth at Uber and Stripe (organization size changed from 200 to 2000) and I think the insights and experiences accumulated in the chaotic environment must be valuable for teams which will face the same kinds of growth and problems embraced by the growth. https://blog.takanabe.tokyo/en/2017/03/python-machine-learning-chapter3.13.2/ Sun, 05 Mar 2017 02:24:00 +0000 https://blog.takanabe.tokyo/en/2017/03/python-machine-learning-chapter3.13.2/ Motivation Learn fundamental Machine learning skill set Chapter3. A Tour of Machine Learning Classifiers Using Scikit-learn Goal of this Chapter Take a tour though a selection of popular and powerful machine learning algorithms Lean about the differences between several supervised learning algorythm for classification Develop an intuitive appreciation of their strengths and weakness Summary of this Chapter Introduction to the concepts of popular classification algorithms Using the scikit-learn machine learning library Questions to ask when selecting a machine learning algorithm(how do we choose? https://blog.takanabe.tokyo/en/2016/10/fluentd-input-plugin-rds-aurora-slowquery-log/ Thu, 27 Oct 2016 13:43:42 +0000 https://blog.takanabe.tokyo/en/2016/10/fluentd-input-plugin-rds-aurora-slowquery-log/ Overview This fluent input plugin collects RDS Aurora slowquery log with ParameterGroup option log_output=FILE. This plugin fetches only the difference between the latest and previous fetched slow query log. Background There are a lot of RDS mysql slowlog input plugins that collect mysql slowquery logs with ParameterGroup option log_output=TABLE. However if you use RDS Aurora with the option, there are two following problems: 1. mysql.slow_log table is operated with Engine=CSV mysql. https://blog.takanabe.tokyo/en/2015/10/redux-boilerplate-with-react-material-ui/ Tue, 20 Oct 2015 13:57:55 +0000 https://blog.takanabe.tokyo/en/2015/10/redux-boilerplate-with-react-material-ui/ Motivation There is no boilerplate of Redux using React & Material UI when I wanted to created a new web application with the combination. Since I felt it was waste of time to solve its dependency problems among React, Redux and Material UI, I prepared its boilerplate on github. takanabe/react-redux-material_ui-boilerplate · GitHub I wish you would consentrate to develop application by using the template :) About boilerplate A boilerplate for React + Redux + Material UI + ES6 syntax applications. https://blog.takanabe.tokyo/en/2015/03/introduction-of-openstack/ Thu, 12 Mar 2015 13:02:38 +0000 https://blog.takanabe.tokyo/en/2015/03/introduction-of-openstack/ Motivation To organise information, knowledge and experiences of OpenStack. About this article A year has passed since beginning development of an OpenStack infrastructure. There is now a lot of information and knowledge available which I didn’t have when I first started developing the IaaS. Through summarising and and organising gained knowledge and experiences and highlighting with some informative articles, I hope to share information with newcomers to the OpenStack world.