introducing papervision3D
carlos ulloa
www.papervision3d.org
blog.papervision3d.org
wiki.papervision3d.org

open source 3d library or Flash

genesis: Spark conference Amsterdam 2005
-presentation by joost korngold – renascent

2006.12.02 – papervision went open source
-papervision license: MIT license – free for commercial use
-open-sourced so that people could use it

2006.12.10
ralph hauwert – the guy who built he rhino

core team: carlos ulloa, John Grdner, Ralph Hauwert

COMMUNITY has been very important. Feedback has been invaluable

WHY PAPERVISION?
-powerful: Flash 3d is extremely difficult to do well.
-easy to use : people should be able to pick it up quickly, immediately be useful
-production driven design
-high performance realtime 3d rendering
-linear texture mapping per face
-hierarchy, instances, materials management

EASY TO USE
-useful for developers and designers
-designed for Flash
-AS3-style syntax -3d objects should not be more complex than movieclips
-no maths required defaults to using degrees

-you can use your own 3d package
-create and modify without recompiling
-preview your scenes in realtime

COLLADA – data format
-open standard
-XML based
-scene format
-multiple objects and textures per scene

-supports camera, materials, paths, tween & skeleton animation, physics
-originally created by Sony for PS3 and PSP-now property of Khronos

Free plugins available for Maya, 3dsm, softimage XSI, and Blender
-adopted by many commercial game studios, game engines, and Google Earth
-Thanks, Collada!

METAPHOR which we use in papervision
-in a computer, 3d data must be rendered in 3d

1. Scene (stage) -> Objects (thing) -> materials (look and feel) ->
2. Scene -> Camera (viewpoint)

OBJECT
org.papervision3d.objects
displayobject3d -> xyz pos, xys rotation, scale, scale xyz, visible, name, parent, root

3d model
-created by a 3d artist in a 3d package (GENERALLY NOT DESIGNED BY THE DEVELOPER)

planes
-planes moving in 3d, mimicking a 3d object

Primitives
-cube, sphere, cone, cylinder

Skybox
-panorama

Particles
-e.g. stars

Materials
org.papervision3d.materials
Textures
-bitmapdata, MC, library assets, jpg, png, flv
-Photoshop CS3 extended

Cameras
-the location from which the scene is being viewed
org.papervision3d.cameras
extends displayobject3d
-target – a thing the camera follows

ONE LINE OF CODE
-each behavior can be implemented with one line of code
-Flash CS3 component in the works

MORE COMING SOON
animation
-Tim Knip – skeleton animation
-Jim Armstrong – classes for hands, arms, etc.

MATERIALS
-visual quality
-z-flat shading : quick, easy, not the best
-phong shading
-z-flat shading textured implementation
-phong shading

-argh! Too many ways of rendering to copy down!

BumpMapping

COMING SOON
Normalmapping – high poly to low poly without datsa loss
specular maps – reflection mapping
cubic environment mapping – thing the Terminator 2
Mip-Maps
Real silhouettes/outline shading..not cheating using a filter
plugin structure for custom materials
lighting structures
shadow structures
z-buffer(?)

PERFORMANCE
current RC2 speed increase: 20%

better clipping, fogging, depth queuing

Demo reel: HOLY SHIT!!!!!!!!

[also lots of photos of the screen]

Experience Information
Marcos Weskamp
www.marumushi.com

-one number really doesn’t mean anything until compared to another number

wikipedia: information visualization

many different ways of visualizing a dataset.
many different datasets can be included in one visualization

Voronoi diagram — look this up

data can be both useful and beautiful

[.eps printouts of dynamic visualizations]

PARSE
ANALYSE
reduce
organize
learn
VISUALIZE
difference
contextualize
reduce

PROJECT: Wieden + Kennedy (www.wk.com)
used QT back in 2001.
brought in MW for a re-build of their site

anything can be information
in any data source, decide what the information points are, and create the interface based on which aspect of the data is most important

APPLICATION ARCHITECTURE

Justin Lewis, Instrument — www.instrum3nt.com

“restful” : CMA
-RoR /Ajax
-beautiful

[I need to dive into Flex]

EXPERIMENT EXPERIMENT EXPERIMENT!!!!!!!

DIGG API EXPERIMENT

Introducing the Chumby
Steve Tomlin CEO
Duane Maxwell Head of Software Engineering
www.chumby.com

What is Chumby?

-still in Alpha-level prototype
-A device
-a company
-a media (widget) network

-plug powered, wifi connected
-connects to Chumby network
-runs Flash Lite 2.1.1, eventually Flash Lite 3

Flash Lite community
-Chumby team
business partner
flash community at large

Why is Chumby?
-some stuff on internet is REALLY important to us
-we can’t spend all day in front of our computers
-but we want the internet around us all the time
-PCs require interaction and full attention and don’t integrate well into our lives
-smartphones don’t PUSH, they PULL
-think “look at your watch” rather than “browse on a PC”
-we want a lot of info in our lives

Chumby: the religion
-make it inexpensive
-make it powerful
-make it “open”

SO WHAT’S THE DEAL FOR US?
-we sell Chumbys as close to cost as possible
-Chumbys display, they don’t store—always connected to Chumby network
-network grows: more widgets

WHAT’S THE DEAL FOR JOHN WINKELMAN (DEVELOPER)?
-we create audience for your work
unlike mobile phone ecosystems, Chumby is open
-no taxes, no publishers, no carrier certification, no deck placement issues
-i.e. fewer middlemen
-Chumby is viral; Chumby content providers retain total control of their content; think of it as Chumby having a license for our work

WIDGETS
if widgets are good, they should be available everywhere
-virtual Chumbys in MySpace, apple dashboard, cellphones, etc.
-widgets should be everywhere, and Flash is more fun

NEW CHUMBY INFORMATION!!!!!!!
-price $165, +$9.95 s&h
-if it is cheaper, we will lose $$
-no additional fees
-no subscriptions
-accessories (chumBling)
-new classic colors, plus limited editions

-accessories to customize chumby (currently 1 USB port, later probably 2)
[input from stuff talked about in “making it physical” ]
-FM radio adapter
-IPOD

-launch this summer with Flash Lite III
-Flash video & audio
-based on Flash 8

WE NEED YOUR HELP
-create and upload cool Flash Lite Widgets
-stay in touch with us (blogs, forms, wikis)
-move to San Diego and work for us!

CHUMBY TECHNICAL SPECS
3 versions of Chumby
-foo — prototype last august
-katamari — 2nd gen prototype
-ironforge (production)

[photo of lots of screens — too much info to type]

Chumbys talk loudly over LANs

-Chumby doesn’t need to be on network to test new hardware/software

-public widgets are hosted on chumby server, but content that Chumby displays can be pulled from anywhere

-the CHUMBY is going to be freaking cool!

-Chumby native resolution is 320 x 240px, but it is Flash, so it is vector

-ads will be more sponsors, rather than commercials
-“chumBooty” — offer for stuff, ring tone, scavenger hunt clue, etc.
-“advertoon” think YouTube, but with advertising
-ads will keep the bandwidth free.
-possible future subscription for people who don’t want ads at all.
-“we’re trying not to screw it up and make it annoying”
-pin-out for larger LCD? Yes, info on forum.
-video — playing 12fps full screen takes up 60% CPU, on the alpha build of Chumby

-Chumby network — limited to Flash files
-a lot of work has gone into privacy/security, so there won’t be any Benedict Chumbys

Building Casual Games in Flash
Philip Kerrman
philipkerrman.com/fitc/

Built most of these games for MSN messenger.

Casual Game: “Carefree game”

<50 megabytes
try -> explore -> buy

“adver-game”
-either fully sponsored (branded up the wazoo) or maybe a subtle watermark, or an ad you have to see before you play

casual games are not usually done in Flash—more often director/java, etc.

Casual game -> you pay for
advergame -> client pays for it

casual game market is HUGE, >50% women

a casual game will sell for ~$20

try/buy conversion rate:
>2% is a hit
1-2% is the norm
<1% is poor

portals will pay the author ~30%

Advergames
-usually work for hire
-simply skinning an existing game isn’t popular
-wide range of money-making opportunities for free games with advertising
-banner ads
-in-game sponsorship
-pre game ad
-in-game (break-time) ad

ARMS RACE
-more and more advanced graphics
-increased user expectations

XBLA—6m xbox users
-try but — very frictionless
-they say 75k – 300k to produce a game
-certification
-games sell for $10
conversion rates around 30-35%
-better revenue share: 50/50 or better
-gatekeepers

WINDOW LIVE MESSENGER
-250m users
15 unique game users per month
-potentially 30-35m players
30% average yearly growth

-subscriptions
-ad revenue

NOMENCLATURE
Up sell — try version gives you a nag screen to buy the full version

frictionless —

Portals — “publishers” for your casual games.
-big portals take a bigger cut, but tend to be more stable and more trustworthy

badges/achievements — visible “pride” based declaration of your awesomeness at a game

ADAPT YOUR SKILLS
-use the appropriate tools for the target market

TECHNICAL STUFF

1. GAME MUST BE FUN!!!!!
2. user experience
2. user experience
4. user experience

USABILITY
MS Games prototyped using PAPER mockups of all the screens

Casual games often have to live within a framework, which YOU have to adjust to, because THEY won’t change it for you.

You will need to include “ad breaks” functionality in your game

You are living in reality
-lean toward the lowest common denominator

MULTI-PLAYER
-griefing situations
-race conditions
-technical limits
-home-made random seed
-turn-based is MUCH easier
-don’t underestimate difficulty involved
2 player game > single player game * 2

AI
-many different ways of thinking about it

PORTING OF GAMES
-many many many venues in which a game can live

LOCALIZATION — allow for many different fonts, lengths of words
-this could be the hardest/most frustrating part of the development

CHEATS
-think of them as “hints”

-Keep the number of server requests to a minimum.
-synchronizing is difficult, but very important
-if two people perform an action on an object at the same time, before the info is sent to the server
-learn ways to make data smaller so it gets sent faster, especially in situations where a lot of info is constantly going back and forth

GRIEVING — one person quits, the other stays on, expecting to play

Games need to be bulletproof, and they need to talk loudly but briefly to the server

[jigsaw game] — every piece has the same registration point: 0:0. Even if it is visually all thee way across the stage, it is a small visible portion of a big empty movie clip

-international portal games can have a long development process

The Blind Sketchmaker- exploring evolutionary and generative art with Flash
Mario Klingemann – quasimondo.com

Can computers create art all by themselves?

No: Only artists can create art.

[stuff about art as a system of belief]

art happens because our brains want an assignment after the basic work is done:

color – find food
danger – pattern matching
aim/estimate distance
orient/find your way home
-select mating partner
-protect and care for offspring
-learn new things
-get bored

What makes it art? Good networking

Komar and Melamid: “America’s most wanted painting”
www.diacenter.org/km/

Artist battle
http://www.saatchi-gallery.co.uk/showdown/
is it art or not?

If not Art can computers at least create art?
can computers create interesting pictures?
Maybe.

Learning to see
-The more you know, the more you see
-What do we have to look for?
-How can we tell art from noise?
-How can a program “See” after all?

This session = Too much art, not enough generative.

This session has nothing to do with anything. I just wasted an hour.

Well, he pulled it together a little bit in the last five minutes, showing us how his tool creates generative art…based on the “Biomorph” experiments of Richard Dawkins.

It has created some badass art.

Great tool, fantastic experiment, boring session

“Let’s Get Physical” — ambiance as input data
Craig Swan
crashmedia

talking about ideas

“change the way you look at things— and the things that you look at change”

API — (H)API Human Application Programming Interface

flash can be a set of ears — using a microphone
-it knows how loud things are—useful for building environments

eg. “ambient awareness”

everything which you can make Flash aware of an be a variable

VISION
-flash can “see” using the camera
-flash can measure change in video from moment to moment
-e.g. motion sensor recording
with bitmapdata Flash can record video and play it back, non-linearly

detecting motion-
flash can react, real-time with video captures being used to interact with “virtual overlays”

gesture-based interface

grabbing bitmap data from live video, applying live filters for display

find keys for content delivery from the environment. Give Flash eyes and ears. take the input and make use of it.

TIME
time as meta data of real-time input
replay frames, or parts of frames, at different time intervals, or completely out of sequence
videograbber
timescanner
timetunnel

TOUCH
all input devices are CONTACTS — one thing making a connection with another

IPAC — allows the creation of your own controllers for a computer input is interpreted as key presses—WHICH CAN BE USED BY FLASH

EXAMPLE. Wired up a wind chime. The wind outside controls an app running in Flash

EXAMPLE: Ethernet cable — wires hooked up to finger rings which, when contact is made, sends info back to Flash

Conductive thread
conductive paint
conductive marker

you can draw an electric circuit on a wall, or sew it into a garment. Plug it into the IPAC and you have a new input device

LITEGRAF
painting with light drawn from a physical palette

EXAMPLE: Wired up a simple lock and key which could be used to lock/unlock accessibility to a computer

TOOLS:
Arduino
Phidget RFID

RFID detectors/readers can be picked up for $60

The many Flavors of Sensors
breath distance proximity light temperature acceleration noise remote current

Once the input comes in, you can can process that info and send it back into the world in another form

“Teleo”?

wireless flash zipzap remote

Sudden Motion Sensors (inside all mac books — accelerometer)

Blinking lights and buttons — MONOME input device

“OSC” Open Sound Control

two-way communication between flash and controllers

Playing with knobs : MIDI

midi2flash by crashmedia olaf matthers

WIImote to Mac controller: wiili.org “darwinremote” by hiroaki kimura

Flashing in public www.snepo.com
Scotty Weeks www.twelvestone.com
Anthony Eden www.arseiam.com

SNEPO: integrating strange back-ends with slick flash front-ends

“we’re so haptic we could squirt”

haptic — interaction through touching

kiosk, POS systems, iPhone, etc.

KIOSK APPS

RULES OF THUMB:
ATTRACT
-inform user what it is
-inform user how to begin
ENGAGE
-me too
-others will be watching the user doing his thing, and therefore learning
FAT FINGERS
-people tend to hit the whole screen at once by leaning on it when they press
-solution: move buttons to the bottom of the screen
-web metaphors don-t extend too well to kiosks
MAKE BUTTONS BIG AND OBVIOUS
-only button presses—no mouseover, no keyboard
-some allow dragging, but dragging on a touch-screen is iffy, at best
-have actions happen on PRESS, not on RELEASE

ACCESSIBILITY
-visually impaired = high contrast, large fonts
-color blindness
-these are in the public arena, so there is no target audience. Prepare for everyone!
-allow for wheelchair/height challenged. Allow for content to be lowered down the screen

NO FINISH
-just because they start something, doesn’t mean they will finish it.
-allow for proper timeout/reset time, based on complexity of application

EASY BUILD
-how easy is it to build the application?
-should be easy to install on the hardware
-TURN THE MOUSE CURSOR OFF!!!

EXAMPLE: “wayfinder” for Westfield mall
about 1 year from beginning to end
-two months of paper prototyping
-four-five months of development, with two months of refining pathfinding algorithm
-testing, refining—
grand total of about 1 year

Make the application “aware” of its physical location and orientation
each installation comes with an administration mode to “initialize” the installation; e.g. meta-data about physical location of install.

WHAT WORKED

TXS/XMLFS
JSFL
-automated the map-making process for the store kiosk (from the example)

-TEST TEST TEST: Lots of paper prototyping
-logging every piece of user interaction
-make data available to client for subsidiary consideration—advertising, etc.
-TESTERS: replicate every possible human interaction. Try to break it. Account for irrational user behavior
-Snepo used interns who took great joy in pointing out developer mistakes
-discovered that touchscreens aren’t usable by people with artificial limbs — the screen didn’t register the touch
-REMOTE MONITORING
-each kiosk has a transaction server which sends a “heartbeat” back to the the main center. If there is no heartbeat, look for Flash process. If no Flash process, reboot the kiosk.
CLIENT RELATIONSHIP
-public kiosk clients tend to have [1] a lot of money, and [2] not all that much technical savvy, and [3] a lot of equity invested in the project

WHAT DIDN’T WORK

TXS/XMLFS
-not a very ergonomic API for Flash developers
-Pathfinding algorithm was done in Flash — Dijkstra
DYING COMPUTERS
-no ventilation
-overheating
-everything made out of steel
-dead hard drives
SCREEN CALIBRATION
-public touchscreens lose calibration quickly
UPDATING SUCKED
-updating had to be done individually

HOW WE IMPROVED THINGS

ENVIRONMENT
-we quit, then started our own company
GOT RID OF TXS
GOT RID OF XMLFS
CREATED “DEPOT” to replace XMLFS — based on HTTP standards
-pulled Dijkstra out of flash and put it on the server—1000% performance improvement
lesson: Things which require a LOT of processor power may be better shuffled off to the back end. Flash is now just the interface, not part of the logic layer

DISC DEATH and WHAT-NOT
-don’t get bent out of shape about hardware failure
EASTER EGGS

OTHER PROJECTS

TICKETING
CHECK-IN/CHECK-OUT SYSTEM – custom Chinese hardware; had to write drivers form scratch
BLIP NODE
-kiosk interaction software for cell phone. Buy media from kiosk, blue-tooth it to a cell phone.

EXPERIMENTS
-RESEARCH IS VERY IMPORTANT
-haptic technology allows for MASSIVE scope

“ERLANG” language for making kiosks talk to each other through a network

FROM KIOSK to POINT OF SALE
-POS equipment

UPSIDES of TOUCHSCREEN (HAPTIC) TECHNOLOGIES
-bye-bye browser
-you get to define EVERYTHING
-there’s money of be made

HAPTIC TECHNOLOGIES

WHAT TOOLS ARE USED?
-flash executables
-rolled their own .swf wrappers in C or VB

Multi-touch? — installations in public spaces generally don’t require this technology
Error messaging — send a friendly message to the user,and a detailed message

RESOURCES:
kiosk forums

The Art of Encoding
Derrick Ypenburg
www.focusonmedia.com

video properties that affect encoding
-color/movement
-edits/transitions/effects
-frame rate
-display size
-sound ranges
-interlacing
-image quality
-smoothness of playback
-audio quality
-transitions/effects

IMPROVING ENCODED VIDEO QUALITY
-2 pass vs. single-pass encoding
-preprocessing filters
-advanced compression settings
-de-interlacing
-keyframes — not like Flash — snapshot of video which subsequent frames use as basis for compression
-smaller display size — 320 x 240 is about as small as you should need to go
-dropping frame rates — the fewer frames, the more room for detail in each frame

3RD PARTY SOFTWARE
-Flash video encoder is okay, but not great
-Sorensen Squeeze is GREAT
-ON2 FLIX is really good too

CODECS
-Spark Pro
-On2 VP6 Pro
-Squeeze Pro

FEATURES
…etc.

Files for this session available at www.focusonmedia.com/fitc2007

(Notes from Flash in the Can conference)

Ryan: “They’re trying to sell me something which I have no choice but to buy.”

MIKE DOWNEY
Flash player adoption rates:
FP7: 75% in 12 months
FP8: 94% in 12 months
FP9: 84% in 9 months

PaperVision demo
-rhino
-xwing obstacle course

Flash Player 9:
-now has full-screen video support. Not full-browser — FULL SCREEN
-10 million downloads a day

CS3 is now shipping
All CS3 products now have a standardized interface

Flash CS3:
-timeline is now in its own panel
-can now import Photoshop files into Flash IDE
-selectively import layers from PS
-multiple options for importing text from PS
-jpeg compression engine is now using the Fireworks engine

-Can now (natively) import Illustrator files into Flash, rather than needing third-party library

Robert Penner stuff
-timeline animations can now be exported as Actionscript/XML.
-this can be done per-layer
-Actionscript class which interprets the outputted XML is included with Flash CS3

TED PATRICK : FLEX
-Flash CS3 can be used to build Flex components
-timeline and “states” integration
-event integration
-download MPX plugin to allow Flex app export from CS3
CTRL+ENTER export of .swc export (.swc == flex component version of .swf)
-a flex component can be ANYTHING

KEVIN TOWES : FLASH MEDIA SERVER
-intelligent streaming
-can detect bandwidth
-can detet player version
Akami — “tour of California” video — streaming 60 gigabytes per second through the media server
-awesome demo of LIVE ENCODING! No Delay! Real-time encoding of webcam feed into .flv ready for streaming.
-standard flv playmack component has been significantly improved
[demo of live video inside flash movie]
-media server can record live video as it happens and save it to the server.
-mention of photobucket.com, which has a tool which allows you to assemble the stuff you have up on photobucket.com into a .flv, rearrange, add effects and titles, etc.

APOLLO
-Deploy Rich Internet Applications as desktop applications.
-Company called “effective UI” created a desktop app for eBay.
-Apollo runtime includes: —Flash player —WebKit HTML engine (the one used in Safari)

-DEMO: Apollo RSS reader — Built using AJAX (the Yahoo! library). Cool stuff!
-Apollo currently available in public alpha www.adobe.com/go/apollo
-plugin coming for Dreamweaver and Flash to natively publish Apollo apps.

ADOBE MEDIA PLAYER
-sneak peek of Alpha build
-RSS-aware
-a lot like the Democracy player
-fully skinnable
-RSS feed can deliver colors, background, ads, content, thumbnails, the whole bit.
-demo of Reno 911 clip, pulled from RSS along with Background image and embedded ad.
-seems to have been built with Apollo…?

MORE APOLLO STUFF
-allows file I/O
-local data storage
-custom chrome
-system notifications and alerts
-multi-window support
-drag/drop
-copy/paste
-apps can run in background
-network API
-hypothetical example: eBay app. You bid on a product, tell the app to run in the background, then get a window alert if someone out-bids you.
-finetune.com? findtune.com?—sneak-peek at desktop player built in Apollo

TED PATRICK
-preview of Flex Builder III
-Flex is for building Applications; not so much about websites or experiences.
–sliderocket.com — Flex-based, online presentation creator

-BUZZWORD: online competitor for Microsoft Word

“Moxie”: Code-name for Flex III SDK and Builder — lots of work being done to make Flex “back-end neutral”; should work equally with all middleware

technologies

(Notes from Flash in the Can conference)

“Based on a true Story”, by Hoss Gifford, www.flamjam.com

Engaging speaker but (by his own admission) a disorganized wanker.

Did cool work for “Liberated Theatre” in London (http://www.liberatedtheatre.co.uk/chorus.htm)

narrative is what provides meaning and context for your work

“You don’t hire an artist and tell him how it should be done. You hire him for what he does”

[interesting project: environment which changes based on the time of day as pulled from the computer on which the project is running]

FLASHTERBATION

www.newbirth.org

site created by

www.sharperfx.com

[BOOK TO FIND: The Long Tail]

the context in which we apply our skills, rather than the specifics of our skills, is what gives us our job titles

Flash animator + big screen = film-maker
Flash animator + DVD = multimedia presentation creator
etc.