Create your own local meeting assistant using Vibe Coding for beginners - with a complete practical tutorial

I wonder if anyone else has this feeling: when a meeting lasts a long time, it's easy for your mind to drift off uncontrollably.

While trying to keep up with the speaker's train of thought, I hurriedly took notes. I nodded frequently during the meeting, but afterwards, when I tried to recall the details, I had forgotten them all...

Although there are now many meeting recording and text conversion tools available, eitherfreeThe time limit is relatively limited. Either long recordings are slow to process, or the most troublesome thing is that after receiving the transcript, I still have to spend time organizing the key points of the meeting and extracting the tasks to be done.

This weekend, I simply used Vibe to code a locally running meeting assistant myself.

This is a blog recording of a multi-person conversation that I downloaded. The meeting assistant not only helped me transcribe it verbatim, but also... AI The key points of the meeting were analyzed, and the viewpoints of the four speakers and suggestions for the next steps were extracted. The whole process took less than a minute!

I've used it to organize meeting content, listen to lectures, blogs, and analyze long videos, and I'm quite satisfied with the results. The entire development process has also been very...SimpleIt can be roughly divided into 4 steps:

• Analyze requirements
• choose Vibe Coding tool
• Turn ideas intoPrompt wordsGive AI Write code
• let AI Fix bugs and improve functionality

In this article, I will explain the entire application setup process and tool configuration. AI I'm sharing this prompt with everyone without reservation. If you also want to completely say goodbye to those expensive subscription-based software, even if you've never written code before, you can follow this process to develop your own applications.

My requirements for a meeting assistant are as follows:

• I directly upload the meeting recording, and the ASR model will convert the speech into text and distinguish the speakers.
• After receiving the manuscript, use LLM Large ModelAnalyze the key points and organize the core content.

Select Tools

When developing applications using Vibe Coding, choosing the right tool is 80% of the battle won. I use Vibe Coding. Claude Code The Step Plan, paired with the leap to the stars.

Claude Code is responsible for writing code, fixing bugs, and debugging APIs; Step Plan provides model capabilities, and importantly, Step Plan has a comprehensive set of model capabilities, including step-3.5-flash and step-3.5-flash-2603 suitable for content analysis, as well as stepaudio-2.5-asr specifically for speech recognition. This perfectly meets my development needs.

Step Plan dosage is still based on Prompt It's based on the number of uses, with a 5-hour limit of 100 prompts and a weekly limit of 400 prompts, making it very cost-effective.

Looking at the overall cost, I spent half a day frantically debugging and developing the entire application, and I didn't even use 25% of my weekly allowance.

I tested several audio clips, about 40 minutes long, and the API cost was only 0.135 yuan, meaning the subsequent usage cost is almost zero.

Vibe Coding Process

After selecting the tools, we will...Prompt wordsSend to Claude Code starts generating code.

Prompt wordsI need to develop a local MVP application for meeting recording to text and meeting analysis. Please help me build the project from scratch and write the code.

This application is installed or runs on the user's local computer, allowing the user to open and use it locally. Audio files, meeting minutes, transcription results, and analysis results are all preferentially stored locally. Speech recognition and meeting analysis capabilities are performed through external API calls.

The application goal is: a user uploads a meeting recording, the system calls the ASR API to transcribe the recording into text, and tries to distinguish between different speakers. After transcription, the system then calls... LLM The API summarizes and analyzes the meeting content, generating meeting themes, core conclusions, to-do lists, risk points, points of contention, key points of each speaker, and next steps recommendations.

The technology stack uses Next.js, TypeScript, and Tailwind CSS. The first version is a local web app, running on localhost. Data storage uses SQLite. Audio files are stored in the local uploads directory. Further encapsulation is possible later. Electron Or the Tauri desktop application.

Please implement the following core functionalities: Users should be able to view a list of past meetings upon entering the homepage, create new meetings, and upload audio files. Uploaded files should support mp3, wav, etc.m4aThe audio file is uploaded in mp4 format. After uploading, the backend saves the audio file to the local uploads directory and creates a meeting record with the status showing as "processing".

The backend needs to encapsulate the ASR calling module, which can be named lib/asr.ts. The ASR API provider, API Key, Base URL, and model name are all read from .env.local for easy switching between different ASR services later. Environment variables include ASR_PROVIDER, ASR_API_KEY, ASR_BASE_URL, and ASR_MODEL. ASR return results must be uniformly converted to the project's internal format, with each segment containing speaker, startTime, endTime, and text. If the API cannot temporarily return the speaker, the transcribed text should be retained and marked as Speaker 1 by default.

The backend also needs to be encapsulated. LLM The module can be called, and the file name can be lib/llm.ts.LLM The API provider, API key, base URL, and model name are all read from .env.local, which facilitates switching between different APIs later.Large ModelEnvironment variables include LLM_PROVIDER、LLM_API_KEY、LLM_BASE_URL、LLM_MODEL.LLM After receiving the complete transcript, a stable JSON output is required, including the meetingTitle, summary, keyDecisions, actionItems, risks, disagreements, speakerInsights, and nextSteps. The actionItems section should include the task details, responsible person, deadline, and priority. The speakerInsights section should summarize the speaker's main points, concerns, and attitudes.

The front-end requires three main pages: a homepage with a list of meetings, a meeting upload page, and a meeting details page. The meeting details page should be divided into full text transcription and...intelligentAnalyze two regions. Transcribe the full text and display it chronologically, showing the speaker, timestamp, and text content. Speaker names should be editable, for example, changing Speaker 1 to Zhang San and Speaker 2 to Li Si.intelligentThe analysis section summarizes the regional presentation meeting, including key conclusions, to-do items, risk points, points of contention, speakers' opinions, and recommendations for the next steps.

Please pay attention to the local application experience. After uploading, a "processing" status should be displayed; if ASR fails, a clear error message should be displayed.LLM Even if the analysis fails, retain the completed transcription results. Do not lose the entire meeting record due to analysis failure. The page style should be clean and simple, suitable for work tools, and key information should be easily understood.

Please generate a complete project structure, including package.json, SQLite initialization logic, environment variable example file, API routes, ASR encapsulation, etc.LLM Encapsulation, local file saving module, database read/write module, type definition, README startup instructions.

Create a complete MVP project that can run locally. After completion, please check for TypeScript type errors, routing errors, environment variable read errors, file upload logic, and SQLite storage logic. Finally, tell me how to install dependencies, how to configure `.env.local`, and how to start it locally.

For beginners who can't understand code, the next step is to type Enter repeatedly in the terminal and select Yes. Claude Code automaticCreate files and write logic.

In less than ten minutes, the prototype of the application was running. The minimalist interface was very clean and pleasing to the eye.

I excitedly uploaded a recording, only to be met with an error message.

Even if we don't understand the code, don't panic when we encounter errors. We can simply send the red error message from the front-end page to [the relevant authority/organization]. Claude Code.

Why does creating a meeting fail?

Claude Code ClubautomaticInvestigate and fix the problem.

I tried again, and this time the meeting was created successfully, but it got stuck again at the audio processing step. And I discovered an awkward problem: all the test recordings I uploaded were still there, resulting in a bunch of running meeting tasks accumulating on the homepage…

So, I temporarily let Claude Code has raised a new requirement:

Add the ability to delete meetings.

As expected, five minutes later when we clicked on the meeting again to view the details, a delete button appeared in the upper right corner of the page.

After cleaning up the junk data, we continued to solve the bottleneck problem. This time, the attempt to transcribe audio into text was successful, but...intelligentThe analysis results show: Transcription successful, but analysis failed: Error: LLM The returned content is empty.

At first, I thought it was an error in one of the interfaces. After checking around, I found that it was probably because the audio was too long, and the transcription and analysis took too long, making it easy for a single request to time out.

So I let Claude I checked the code to see the limits on the duration, size, and number of words for transcribing imported audio files. Sure enough, the problem was with the API request duration.

Claude The code response indicates that a single audio session should ideally be 10-30 minutes long, with a corresponding 5000-10000 characters for the transcribed text. If the audio is too long, it needs to be sliced beforehand.

Since it's not a model capability issue, let's continue optimizing. After all, daily meetings typically last more than 30 minutes, and it would be too cumbersome to pre-process them every time.

Change long audio processing (over 10 minutes) to asynchronous tasks + segmentation. The upload API should only return the jobId; avoid making the frontend wait indefinitely for the request to complete. The backend should asynchronously slice, transcribe, summarize, and merge the results based on the jobId. The frontend should poll the jobId status, displaying "Uploading," "Slicing," "Transcription," "Summarizing," "Completed," and "Failed." Retain the `maxDuration` configuration, but avoid relying on a single API request to run for an extended period.

This change took much longer, but we still chose Yes all the way through.

This time, I uploaded 78 minutes of audio recording material directly, and the meeting assistant handled the transcription and analysis in one go, with very clear results.

Moreover, I found that it is not only suitable for taking meeting minutes, but also for learning about and studying various audio content, such as video interviews, podcasts, and so on.

From uploading audio, calling ASR, generating transcripts, to calling...Large ModelSummarizing meetings, extracting tasks, and analyzing speakers' viewpoints—in the past, this would have been at least a complete task. SaaS The core functionality of the tool. But now, with the help of... Claude With Code and Step Plan, even an ordinary user can quickly build their own workflow on their local computer.

More importantly, it's not limited to running only one demo. I can continue to add features based on my own usage habits, offering virtually unlimited freedom.

The real value of Step Plan lies in reducing the trial-and-error costs of Vibe Coding.

The most time-consuming part of developing this kind of local meeting assistant is actually the repeated debugging. Interface errors need to be fixed, and ASR return formats need to be adapted.LLM Analysis failures need to be investigated, and long audio clips require further optimization. If all charges are based on the standard API, users will easily become hesitant to experiment due to concerns about costs while debugging.

Step Plan is a monthly subscription specifically designed for coding, making model call costs predictable, allowing me to confidently let... Claude Run the code through multiple rounds, make multiple revisions, and try multiple solutions.

For ordinary people, the most important thing is to be willing to try and use things without worrying about costs.

Original link:Vibe Coding in Practice: I Created a Meeting Assistant in Half a Day