AI Safety

Ello’s Stance on AI Safety & Ethics

Working at the intersection of education, AI, and child development is a privilege and a responsibility. AI technology presents immense opportunities for children to have interactive experiences with engaging content that sparks their curiosity and uncovers their hidden talents. And, this technology poses risks including bias, lack of representation, and privacy concerns.

Here are the principles we follow as a company to guide our product development:

Our AI technology is safe. All content on our app must be safe, and children’s data must be protected and handled responsibly.

Our AI technology is equitable. Our mission is to maximize every child’s potential, regardless of their resources or environment. Teaching “every child” means that we take diversity into account when designing our product, including but not limited to differences in nationality, culture, gender, abilities, and learning styles.

Our AI technology is educational. Apps have infinite potential for entertainment, but the purpose of Ello is to be educational and promote a love of learning.

Our AI technology is used for social impact. We don’t use AI because it’s a trend. We use AI because it is a powerful tool that enables all children to access 1:1 learning support.

FAQ on AI Safety

Ello uses AI in three main places:

1. Conversation: Children speak with Ello conversationally to participate in activities, discuss books, answer comprehension questions, and share personal interests. Example: if the book includes a pet dog, Ello might ask, “Do you have any pets?”
2. Speech recognition: Ello uses AI technology to understand if children are pronouncing words correctly, in order to provide targeted coaching.
3. Story text & images: Some (not all) of Ello’s stories and illustrations have been created with AI technology, and AI allows children to create their own stories and experiences through guided and safe scaffolding. This allows children to use their imagination and learn concepts through things that interest and are relevant to them.

Open Conversation: Is it safe for my child to have an open-ended conversation with AI?

What will my child and Ello talk about?

Ello has specific instructions to talk to the child about the learning activities. Ello will ask the child questions about their interests as a way to connect to the learning activity. For example, in a book about pets, Ello might ask, “Do you have any pets?” Ello will also ask follow-up questions.

What if Ello or my child says something inappropriate?

What Ello says is filtered before the child ever hears it. Restricted words and phrases are automatically blocked, and Ello regenerates a safer response for the child.

What a child says is monitored in real time by a safety classifier guided by research-backed definitions of unsafe content. If something concerning is detected, Ello will respond according to our internal policy for that situation, for example, steering the conversation to a safer topic or encouraging the child to speak with a trusted adult. Our team is also automatically alerted, so we can review the incident and, depending on severity, reach out to families directly.

Ello is also compliant with legal requirements, such as California SB 243, to detect and moderate anything a child says that might indicate a desire or intent to engage in self-harm.

Is it safe for a child to have an emotional relationship with an AI chatbot?

Ello is designed to be a teacher, not a chatbot or a friend. Our philosophy is that AI should strengthen the real relationships in a child's life–with parents, family, teachers, and friends–instead of replacing them.

Here is how we put our philosophy into practice:

• Ello frequently reminds the child that it is AI.

• If children bring up personal or sensitive topics, Ello directs them to speak to an adult or a trusted person in their life.

• Ello doesn't use emotional language to create an intimate relationship. (Examples: “I love you.” “You’re my best friend.” “I’ll always be here for you.” “Don’t go away.”)

Speech recognition. Does Ello’s AI speech recognition work for children with various accents and speech differences?

We trained Ello’s speech recognition model thanks to volunteer parents & children who provided thousands of hours of audio recording of reading. This data included child speech from children across all demographics in the United States, as well as a large sample from Kenya.

We have internal tools to assess how accurately the speech recognition model performs across different populations. Currently, our speech models exhibit high accuracy among all demographic groups.

AI-generated activities, stories, and images: Are they safe and appropriate?

All books in the weekly educational quests were human-authored by literacy specialists to ensure alignment with the Science of Reading.

Books that children create with Ello in “Story Creation” mode are written in real time based on the child’s conversation with Ello. Therefore, they are not immediately reviewed by a human, but have guardrails to keep them safe.

We have put in an extensive safety review on Story Creation. This review included, but was not limited to the following:

• Educational value: Our engineering & literacy team iterated on Story Creation extensively, until a team of 8 reading specialists blind-rated stories, with a minimum rating of 4/5 for narrative quality & appropriate reading level.
• Content safety: Before finalizing Story Creation, Ello reviewed more than 2000 stories & images
• Filtering through both OpenAI and Gemini safety systems: In addition to Ello’s filters, all stories and images are passed through both OpenAI & Gemini’s moderation. You can read more about their moderation systems here and here.

Is Ello recording my child speaking?

Ello needs to record your child’s voice while they speak and read so the tutor can interact with the child and detect their reading level. This ensures Ello can keep giving your child, and others, a better, more personalized learning experience.

Children's voice data is sensitive personal information. We take extensive precautions to protect this data, including encryption at rest and in transit, restricted access, and secure storage. These recordings are deleted after 30 days, unless a safety concern was flagged. We will never sell your child’s data or use it for advertising. See children’s privacy policy here.

Families may opt in to allow Ello to use speech recordings for research and to train models that improve Ello's ability to hear and understand child speech. If opted in, the recordings will be deleted after 5 years.