Story-to-handdrawn-video test - One-click generation of hand-drawn picture book videos

Today I'm sharing some things that are suitable for making story picture books.

Creating a picture book is usually a very complicated process, involving splitting the frames, processing each raw image, handling subtitles, creating black-and-white to color animation, setting transitions and exporting parameters. The process is very fragmented.

Today's name is story-to-handdrawn-video This GitHub project integrates all the processes of creating picture books into one comprehensive system.Prompt wordsThat's how you can produce a work.

I ran a video with sound added using CapCut, and the effect was truly amazing:

Case 1 Osmanthus Postcard

Prompt words：

Use $story-to-handdrawn-video to turn the following story into a hand-drawn diary comic animation.

Task settings:

– Title: "Osmanthus Postcard"

– Generate images using the default Codex Image2

– Generate captions using project fonts

– Use the cut transition

– First, complete image generation, material import, and preview rendering.

– Output 720×960, H.264 silent preview

– Retain the original story text, with each complete sentence serving as a storyboard.

– The illustrations maintain a pure white background, hand-drawn black lines, and low-saturation crayon colors.

– All people and items must be present intact within the safe zone.

– No need to add voiceover, music, speech bubbles, or on-screen text.

– After completion, copy the preview video as:

out/tests/guihua-postcard-preview.mp4

– Report the number of storyboards, video duration, and final file path.

Character locked:

Aning, a 26-year-old woman, has shoulder-length black hair, wears a beige short jacket, a light blue long skirt, brown canvas shoes, and carries a dark green canvas bag. She maintains the same face shape, hairstyle, clothing color, and body proportions in all scenes.

An elderly man, around 70 years old, with short white hair, thin, wearing a gray Chinese-style jacket and dark blue trousers. The old man only appears in the frame after the story is clearly established.

Main story:

Early in the morning, Aning found an unfinished postcard in an old book.

The postcard had only one sentence written on the back: "I'll go home when the osmanthus flowers bloom."

Aning walked along the old address to the south of the city and saw that the osmanthus flowers next to the courtyard gate had already bloomed.

An elderly man with white hair took the postcard and lingered on the line of text for a long time.

In the evening, when Aning left, the old man handed her a small bag of osmanthus flowers.

The protagonist's hairstyle, blue dress, and green canvas bag are well-preserved. The old books, postcards, courtyard gate, and osmanthus tree create a clear progression of the scene. The subtitles remain readable throughout, and the characters and buildings are not cropped. The final handover of the osmanthus flowers is also complete, and the story concludes smoothly.

Case 2: The repaired yellow umbrella

Prompt words：

Use $story-to-handdrawn-video to turn the following story into a hand-drawn diary comic animation.

Task settings:

– Title: "The Repaired Yellow Umbrella"

– Use the default Codex Image2

– Subtitles use the project font

– Use the cut transition

– Perform complete image generation, import, and preview rendering

– Retain the original text; one complete sentence corresponds to one storyboard.

– Yellow umbrellas must always maintain the same color, curved handle, and location of any damage.

Uncle Zhou's dark blue toolbox must maintain the same appearance.

– The illustrations are in a hand-drawn diary style on a white background, with plenty of white space.

- Watermarks, signage text, dialogue bubbles, and extraneous figures are prohibited in the footage.

– After completion, copy the preview video as:

out/tests/yellow-umbrella-preview.mp4

– Report the number of storyboards, total duration, and file path.

Character locked:

Uncle Zhou, a 55-year-old male, with short gray hair, a square face, wearing a dark blue repairman's uniform and black cloth shoes, and carrying an old dark blue toolbox.

Lin Xi, a 20-year-old woman, has a black high ponytail, wears a dark green knit sweater, beige trousers and white canvas shoes, and is holding two books in her arms.

The two characters must have the same face shape, hairstyle, clothing color, and body proportions.

Main story:

After finishing his night shift, Uncle Zhou noticed a girl sitting at the bus stop, carrying a book.

The rain was getting heavier, and suddenly one of the ribs of the yellow umbrella in the girl's hand broke.

Uncle Zhou took out a thin wire from the toolbox and bent down to wrap the umbrella ribs tightly.

The last bus arrived, and before boarding, the girl placed an orange next to her toolbox.

The next morning, Uncle Zhou opened his toolbox and found a note that said "Thank you" under the oranges.

Uncle Zhou, the girl, the dark blue toolbox, and the yellow umbrella remain constant, while the bus stop, the rain line, the last bus, and the early morning form a complete temporal progression. The orange appears before the girl leaves and remains beside the toolbox the next day; the story leaves no gaps in the narrative.

However, even though one of the umbrella ribs is not clearly drawn, it will still appear as a complete umbrella at first glance.

Case 3: Spring in the Blue Bag

Prompt words：

Use $story-to-handdrawn-video to turn the following story into a hand-drawn diary comic animation.

Task settings:

– Title: "Spring in the Blue Cloth Bag"

– Use the default Codex Image2

– Subtitles use the project font

– Use the cut transition

– First, create a visual-plan JSON with two-digit scene numbers based on the story.

– The visual plan must clearly define Jiang He's age, clothing, location, actions, and the characters who need to appear in each scene.

– Save the visual plan to:

out/tests/mint-visual-plan.json

– Use this visual-plan to continue performing image generation, import, and preview rendering.

– Retain the original text, with each sentence forming a story beat.

– The blue cloth bag, white flowerpot, and mint seeds need to maintain visual continuity.

– Grandma only appears in the scenes explicitly mentioned in the original text.

– Once completed, copy the preview as:

out/tests/mint-spring-preview.mp4

– Report the number of storyboards, total runtime, visual-plan path, and video path.

Character locked:

Jiang He has a round face, slender eyes, a small mole at the end of her left eyebrow, and slightly curly black hair; these features remain consistent throughout her life.

Eight-year-old Jiang He is wearing a light yellow sweater and dark blue overalls.

Sixteen-year-old Jiang He wore a white shirt and gray-blue trousers, and carried a brown schoolbag.

26-year-old Jiang He has shoulder-length, slightly wavy black hair and wears a light gray trench coat and dark green trousers.

My grandmother was about 68 years old, with short silver-gray hair, wearing a dark red cardigan and a dark gray long skirt.

Main story:

When Jiang He was eight years old, she would stand on a small stool every day to water the mint on the windowsill.

Grandma sewed a small packet of mint seeds into a blue cloth bag and handed it to Jiang He.

At the age of sixteen, Jiang He left the small town, taking only that blue cloth bag with her.

Ten years later, Jiang He dragged her suitcase back to the old house, where the windowsill was covered in dust.

Jiang He planted the seeds from the blue cloth bag into the original white flowerpot.

When spring came, a new mint plant sprouted in the flowerpot.

The characters Jiang He at age 8, 16, and adulthood show age changes, while her slightly curly black hair and round face maintain a certain continuity in her identity. The blue cloth bag, white flowerpot, and old windowsill provide stable visual cues.

When the adult Jiang He returns to the dust-covered old house, the empty flowerpots and suitcases perfectly represent the passage of time, and the appearance of the last fresh mint and blue cloth bag at the same time accurately captures the emotional impact.

However, the small mole at the end of the eyebrow is almost invisible in the current art style and character size, and such details are not suitable for identification.

Case 4: The old keychain from the noodle shop

Prompt words：

Use $story-to-handdrawn-video to turn the following story into a hand-drawn diary comic animation.

Task settings:

– Title: "The Old Keychain from the Noodle Shop"

– Use the default Codex Image2

– Subtitles use the project font

– Use the cut transition

– Completely execute image generation, material import, and preview rendering.

– Retain the original story text and dialogue wording

– Each complete sentence corresponds to one storyboard.

– Dialogue is only displayed in the top subtitle area; no dialogue bubbles are generated in the illustrations.

Rain, wet clothes, and hot noodle soup are depicted using simple hand-drawn lines.

– Old keychains need to maintain the same round wooden appearance.

– Avoid exaggerated crying and dramatic physical movements

– Copy the preview after completion:

out/tests/noodle-shop-preview.mp4

– Report the number of storyboards, total duration, and video path.

Character locked:

The proprietress, a 48-year-old woman, has short, curly black hair and is wearing a brick-red apron, a beige shirt, and dark gray trousers.

The deliveryman, a 24-year-old male, with short black hair and a slender build, was wearing a soaking wet light blue delivery raincoat and black trousers, and was holding a white helmet.

All scenes maintain consistency in character facial features, clothing color schemes, and body proportions.

Main story:

Late at night, as the noodle shop was preparing to close, a deliveryman, soaking wet, stood at the door, rummaging through his pockets.

The proprietress brought out a bowl of hot noodle soup and said, "I cooked too much noodles, don't waste them."

The deliveryman was eating noodles with his head down, raindrops from his helmet falling to his feet.

Before leaving, the deliveryman placed an old keychain on the table and said: Come back to redeem it tomorrow.

The next day, as soon as the proprietress opened the door, she saw a bag of hot soy milk hanging on the doorknob.

The old keychain is still placed in the most conspicuous place on the counter.

The atmosphere at the noodle shop entrance—rain, wet clothes, and hot soup noodles—was wonderful. The owner's red apron and the deliveryman's blue raincoat remained consistent. The order of actions—heating the noodles, eating with head down, putting down the food, and opening the shop the next day—was clear.

But the old keychain looked like a brown disc, more like a coin.

Case 5: A watch stopped at 6:10.

Prompt words：

Use $story-to-handdrawn-video to turn the following story into a hand-drawn diary comic animation.

Task settings:

– Title: "The Watch Stopped at 6:10"

– Use the default Codex Image2

– Subtitles use the project font

– Use the cut transition

Keep the original story text; do not shorten sentences in advance.

– Run the plan mode first and report.automaticSplitting results

– After the report is completed, continue with image generation, material import, and preview rendering without waiting for confirmation.

– Each storyboard subtitle can have a maximum of three lines.

– The blue collar, old watch, and brown puppy must be consistent.

– The white-haired woman only appears in the scene after the phone call is connected.

– Once completed, copy the preview as:

out/tests/watch-at-six-ten-preview.mp4

– Report the number of original sentences, the number of split scenes, the total duration, and the file path.

Character locked:

Mr. Xu, a 62-year-old male, with short gray hair, wearing round-framed glasses, an old brown vest, a light gray shirt, and dark trousers.

A white-haired woman, approximately 65 years old, with ear-length white hair, wearing a dark green coat and a black long skirt.

The puppy is a brown short-haired dog with its right ear folded down and wearing a blue collar. A silver, round, old watch hangs from the collar.

Main story:

In the evening, watch repairman Uncle Xu closed his shop door. He had only taken two steps when he heard a soft scratching sound coming from under the counter.

Uncle Xu bent down and saw a small dog wearing a blue collar with an old watch that stopped at 6:10 on the collar.

Later, Uncle Xu dialed the phone number on the back of the watch. A white-haired woman answered the phone, and after a few seconds of silence, she burst into tears.

Half an hour later, the woman hugged the puppy, and Uncle Xu handed her the repaired watch, the hands of which had moved back to 6:10.

Uncle Xu's glasses, brown vest, and watch repair shop remain stable, while the brown puppy and blue collar show no obvious signs of drifting. The story unfolds with all the necessary elements: closing the shop, finding the puppy, making a phone call, the woman crying, and returning the watch.

automaticThe number of shots after splitting is reasonable, and long sentences are not crammed into the same subtitle.

Overall, the art style and color scheme of story-to-handdrawn-video are very good, and the storyline flows smoothly. However, if key items are too small, they are not well represented.。

story-to-handdrawn-video Take the Chinese story text or a set of ordered images and create a 3:4 vertical hand-drawn comic animation. The finished product uses a fixed visual syntax of white background, handwritten subtitles, and black and white line art converted into color illustrations. It is output as an H.264 silent video, with voice-over and music left for post-production.

Project address: https://github.com/gnipbao/story-to-handdrawn-video

Project positioning is a set Agent The production process covers story segmentation, storyboard planning, drawing tasks, material cutting, animation arrangement, and video export.

story-to-handdrawn-video Suitable for family stories, children's stories, diary comics, and WeChat official account story videos with a fixed style; also suitable for converting a batch of generated comic pages.fastConvert into silent material with revealing animation.

The project currently only supports single-image storyboards, a fixed white background hand-drawn style, a 3:4 canvas, and silent output. Lip-syncing, character animation, camera movement, voice-over synthesis, subtitle timelines, 9:16 final cut, and multiple art styles require further development.

story-to-handdrawn-video The workflow, which includes one beat per line, character reference, black-and-white to color conversion, and silent export, is designed for repeated execution. The current version is suitable for a wide range of content production.

The installation process will directly provide the GitHub link.AI AgentSimply let it install this project for you.

It's best to choose the one with the raw image model when using it. AI Agent That way you can have onePrompt wordsThe video was generated in one go.

Short videos and text/image content are increasingly reliant on stable and continuous visual production.

For individual creators, small studios, and content teams, writing the story is only the first step. The real time-consuming part is the constant switching between storyboards, raw images, subtitles, animation, and export.

Story-to-handdrawn-video organizes these scattered steps into a repeatable workflow. As long as character settings, scene changes, and key props are clearly defined beforehand,AI Agent This allows you to continue with storyboarding, illustration generation, black-and-white to color animation conversion, and video export.fastProduce a sample version that can be viewed directly.

This workflow is suitable for testing WeChat Official Account Stories, children's picture books, family-themed short films, diary comics, brand stories, and IP content. It can reduce software switching, shorten the drafting cycle, and verify the story direction in batches before deciding whether to invest in voice-over, editing, and detailed animation.

story-to-handdrawn-video At this stage, it is more suitable to create the initial version and silent material. Voice-over, music and complex shots will need to be added in post-production.

Original link:Low-starred gems on GitHub: turn stories into picture books in five minutes.