Font support, adding glowing highlights to text with read-aloud, etc.

In another section of the forum, user eReadingbook wrote:

"I have 4 apps in the app store (all under eReading) all of which can highlight words in a sentence as the sentence is spoken. I would like to be able to have that feature included as I cater towards older children with reading issues. This way they could listen (without highlighting), listen (while it is being highlighted), or click on the individual word and have it spoken. It looks like you can do some of that right now, but it looks very time consuming as I have a lot of text to code. I use real voices not robo voices and in my apps I have to code each individual word. I am looking for something more user friendly and faster."

Hi eReadingbook, this is an awesome question or request and I think it deserves its own heading: access to more fonts in Composer and better support for text effects is something that came up in beta testing and I think will continue to be an important need for book designers.

Right now, there's only limited support for text boxes; you have about 10 or so fonts to choose from, and you can change color, size, and justification, but that's about it. What I have heard from some folks is that they are doing the text in Photoshop and importing the text and glow layers as image objects into Composer, then making each line or each word a touch behavior. If you used the image layers as an animation, you could time the animation to go along with a voice track. It would take a while if you had a lot of text to prepare the images and then import, position, lock and assign behaviors, but it's doable.

The only way I can think of that they could add highlighting as a feature, would be if they created a special font that had a glowing and non-glowing version of each letter, and then somehow grouped the letters into words as a single object, then added a glow/non-glow switch to the word object and you could time it out with wait behaviors timed to the audio track. I think it would be easier and probably look better to just do all the text and glow effects in Photoshop and import the assets individually.

Many many interactive books have this feature, and I realize it's important; there may already be a plan in the works to support this as a feature. In the meantime, I know that most pro video editing software has a titling component that lets you add a glow to text; I have this in Sony Vegas Pro and Adobe Premiere Pro CS5 and I think it's also in Final Cut Pro. Those also let you save a jpeg image of the screen, so it would be easy to turn the glows on and off and save jpegs quickly, then cut those down in Photoshop. I know Photoshop has the feature also, but I'm less familiar with it.
1 person likes
this idea
+1
Reply
  • I tried to have the audio sync with text glow on/off. The problem was that the audio files can't be broken apart and still sound natural. I'm hoping this will work when we can add video.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. indifferent, undecided, unconcerned kidding, amused, unsure, silly happy, confident, thankful, excited sad, anxious, confused, frustrated

  • John and Anna, appreciate the feedback. We are looking into better (or more efficient) text objects that link with narration and/or audio. We would appreciate any feedback you have in terms of what you are trying to achieve as this would help inform our development. Specifically, what effect for the reader are you trying to create and what adjustments would you like to be able to make to a narration effect or object?
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. indifferent, undecided, unconcerned kidding, amused, unsure, silly happy, confident, thankful, excited sad, anxious, confused, frustrated

  • Right now, I'm not trying to link audio narration with text, but this is such a common feature of interactive books, I think it will be something that will be in great demand. As the father of a 6-year-old who is learning to read, I see great value in interactive books that are actually used in helping children recognize words and connect words with the correct pronunciation.

    So with relation to interactive books, I look for books where there is an option for the book to read aloud to the child, and the words are highlighted as the book reads the words; but many such books also offer an option, once the child has heard the story, to turn off the narration and try it themselves. But these books will also light up each individual word when touched. Programming-wise, this seems pretty complicated.

    You have to build the read-along version so that the words light up as the audio is read; the only commercial software that I can think of that has anything like this sort of functionality is CrazyTalk Pro, which will animate a digital "mouth" to look like it is actually saying the words in the audio file, and what it does I think is to pair a set of mouth animations with the sound file by voice recognition.

    I think creating a program that would actually recognize words, add a digital "highlight" to the text with the speech etc. automatically would be really complex, and very expensive, and way beyond what Composer can currently do, or really what you would want to do with it. It's just too much development for one feature.

    However, I can think of a possible feature that would have multiple uses that would accomplish the same thing, although I can't imagine how you would make it fit within the current simple UI of Composer: an Audio timeframe that lets you load an audio file and create a new audio object that is also actually a stand-alone playlist that contains discreet behaviors embedded into it. This way, the "add wait" would be built into the timeline playlist; essentially it would be a graphic representation of "add wait". You load in the audio file, and you can see the audio waveform, then you can play it and select a point on the timeline to "Add Behavior X" which would then go into the regular behavior dialog and let you select an object to attach a behavior to. This would make it really easy to find the spot where you want a word to highlight, create the behavior, move on to the next spot, etc., until you are all done with the text on that page, then save the entire audio object playlist as a packet of nested behaviors that would execute as the audio file played.

    Now, in actuality, you can do this now, simply by sketching out the whole string of behaviors and doing the highlighting and "add wait" behaviors in order, and just use an audio editor to find the values for each "add wait." If you work with an audio editor, you can simply write down the time code of each word as it is read, then program the page behaviors list with the series of "hide/unhide glow" for each word, then add a wait time, then the next hide/unhide glow for the next word, then the next wait time, etc.

    There's a wonderful free audio editor called "Audacity" that works for Mac or PC, and it works very well for recording audio; I would recommend any book author to get it and use it with your audio files and just use the current functionality of Composer to build these complex audio/hide-unhide/wait Dagwood sandwiches of behaviors to achieve the read-along effect, and just build your text blocks and the corresponding glow versions of each word in Photoshop and import into Composer through iTunes.

    In terms of saving development time, it would be a lot cheaper to just provide user documentation and a few videos to educate the Composer user on how to achieve these effects using the current functionality, rather than spending valuable resources that could be spent on adding more unique functionality to V2, in my opinion. I will be glad to help with this project and will add this to my list of upcoming videos, since it will probably be something a lot of folks will want to do. This is something where an author/illustrator/Composer programmer will just have to do the hard work themselves to get the result they want, and with a little guidance I think it's perfectly achievable with the product you already have.

    I think for anyone getting involved with Composer, it's important to realize that with the iPad we are taking a step back in time to the hardware limitations and functionality simplicity of the mid-1990's. Composer is more like "Wordpad" or "Notepad" in terms of word processing capability than the current version of Word or Pages with their multiple automation features. Building interactive books is a bespoke process, and in order to get away from formulaic products, it's necessary to have to do a lot of "from the ground up" construction rather than expecting Composer to provide a ready-made solution for every effect we might like to replicate.

    Sorry if this sounds a bit like a sermon, but if people only realized what a monumental achievement Composer is in its current form compared to the barren landscape that existed for anyone trying to do app development without knowledge of iOS, they would just be grateful that it offers as much as it does and work around its limitations. Sure, a programmable audio/object timeline would be great, but honestly that's software one would expect to find on a PC or Mac workstation, not on the iPad itself. I say lets keep Composer simple and powerful and keep adding features, rather than trying to make it do more than the iPad is really capable of handling.

    That said, it's now incumbent on me to illustrate the procedure I outline above so that folks can get busy building the kind of books they want to make! :)
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. indifferent, undecided, unconcerned kidding, amused, unsure, silly happy, confident, thankful, excited sad, anxious, confused, frustrated

  • Thanks, John. Everything you say here makes a lot of sense. Since the Demibooks team is working on the possibily of adding video, I've been wondering if it would possible to create a video of each section (page) of text with the words highlighting as the words are spoken. For the individual words, the images would have to be created, highlighted and not, that exactly match the video, and programmed as I have done in Not Without Bear.
    I will be curious to see your results.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. indifferent, undecided, unconcerned kidding, amused, unsure, silly happy, confident, thankful, excited sad, anxious, confused, frustrated

  • Hi Anna,

    Yes it would actually be a lot easier to create the text block as a video with transparency since you could build the video and audio together in the pro video editor timeline. You could build it with transparency and hold on the last frame without the glow, so it could be used as a static image as well.

    The only problem is, we don't know what sort of video controls will be included. Usually a video file appears on a page in which it is embedded as a plain black window, then when it plays it appears as normal, but again there's no way to tell how it would appear on the page when added as a video object. If the video is triggered by user touch, and the touch based individual highlighting of the words is also active, it could create some odd simultaneous playback. Still, under the best circumstances, doing a video for text read along would potentially be more efficient in terms of memory management and quicker than building it with hidden/unhidden behaviors alternating with add wait intervals.

    I honestly hadnt thought of using video for the text, it'll be interesting to try that; thanks for the idea!
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. indifferent, undecided, unconcerned kidding, amused, unsure, silly happy, confident, thankful, excited sad, anxious, confused, frustrated

  • I usually have the page text audio begin on "Page did appear", and use an individual transparent sender for each word. If one wanted to add a replay function of the page text audio, it would be a simple matter to have a replay icon appear when audio-or video- ends.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. indifferent, undecided, unconcerned kidding, amused, unsure, silly happy, confident, thankful, excited sad, anxious, confused, frustrated