Pose estimation methods By KH Wong Pose estimation

Pose estimation methods By KH Wong Pose estimation methods v 01. b 1

Overview • • Introduction Problem definition Linear techniques Iterative techniques Comparisons Implementation issues Conclusion Pose estimation methods v 01. b 2

Introduction • What is Pose estimation? • What is pose estimation used for? Pose estimation methods v 01. b 3

Problem definition • What is pose? Pose estimation methods v 01. b 4

Pose estimation techniques • • • POSIT Epnp (solve pnp in opencv) P 3 P P 4 P P 7 P Linear methods – P 3 P – Power 4 methods Pose estimation methods v 01. b 5

POSIT • Pose estimation methods v 01. b 6

POSIT • Pose estimation methods v 01. b 7

POSIT Algorithm • Pose estimation methods v 01. b 8

POSIT • Because the camera model is SOP scaled orthographic, it is an approximated camera • POSIT is based on POS but add a few iteration to improve the result. Pose estimation methods v 01. b 9

EPn. P: Efficient Perspective-n. Point Camera Pose Estimation • • [1] Barycentric coordinate system https: //en. wikipedi a. org/wiki/Barycen tric_coordinate_sys tem [2] Vincent Lepetit · Francesc Moreno. Noguer · Pascal Fua, "EPn. P: An Accurate O(n) Solution to the Pn. P Problem", International Journal of Computer Vision , February 2009 World corrd. sys Control point j Camera Coord. sys 10 Pose estimation methods v 01. b

Application of Epnp (solvepnp in opencv Aruco pose tracker https: //docs. opencv. org/3. 4. 0/d 5/dae/tutorial_aruco_detection. html • • • From //https: //github. com/njanirudh/Aruco_tracke r Download project. zip , unzip to a dir. e. g. Aruco_Tracker-master Copy Aruco_Trackermastercalib_imagescheckerboard*. jpg to https: //youtu. be/Is. XWrc. B_Hvs? t=6 Aruco_Tracker-mastercalib_images change line 6 of aruco_tracker. py to : – cap = cv 2. Video. Capture(0). – Run it – conda>>pythonaruco_tracker. py Print the 6 x 6 aruco marker (in Aruco_Tracker -masterimagesmarker_66. jpg) and show to the camera, it will be tracked Stem 4: Arduino and Computer vision, v. 0. b 2 https: //www. youtube. com/wat ch? v=s. V 7 v. OTv. UCx 8 11

Barycentric coordinate system https: //en. wikipedia. org/wiki/Barycentric_coordinate_system [1] • Any 3 D world feature (or called model) point Piw ( ) can be represented by N(e. g. N=4) 3 D world control points ( ) Cjw=1, 2, 3, 4 using the formula below =3 D feature point Pwi=1, 2, 3, 4 in world coord. =3 D Control point Cjw=1, 2, 3, 4 in world coord. World Coord. sys Choose the center of the model points as one of the control points Camera Coord. sys Pose estimation methods v 01. b Our pose estimation task is to find pose estimation Rotation (R), Translation (T) between the world and camera coord. Systems if model points (Piw) and their image points Ui=[u, v]i. T are given 12

The Barycentric coordinate parameter calibration If you are given Pwi=1, 2, . . , n and cwi=1, 2, 3, 4 you may recover ij Pose estimation methods v 01. b • 13

Barycentric coordinate parameter calibration If you are given Pwi=1, 2, . . , n and cwi=1, 2, 3, 4 you may recover all ij • Pose estimation methods v 01. b 14

EPn. P problem setting Pose estimation methods v 01. b 15 •

EPn. P basic formulation Pose estimation methods v 01. b 16 •

EPn. P : Our plan to find the pose between the camera and world: find Pcj=1, 2, . . , n, then compute Rotation (R(3 x 3)), Translation(T(3 x 1)) Pose estimation methods v 01. b • 17

EPn. P continue to find Pcj=1, 2, 3, 4 • 18 Pose estimation methods v 01. b

EPn. P continue to find Pcj=1, 2, 3, 4 • 19 Pose estimation methods v 01. b

EPn. P : what to do next after V is found • Pose estimation methods v 01. b 20

How to recover ? Case: N=1 • Pose estimation methods v 01. b 21

How to recover ? Case: N=2 Pose estimation methods v 01. b 22 •

How to recover ? Case: N=2(Continue) Pose estimation methods v 01. b 23 •

How to recover ? Case: N=2(Continue) Pose estimation methods v 01. b 24 •

How to recover ? Case: N=2(Continue) • Pose estimation methods v 01. b 25

Verify which case is better • Use image projection to judge which case (N=1 or 2) can generate the best result with minimum re-projection error • If the re-projection error is still large, Use case N=3 and 4 to find the scaling factors. Details can be found in [2] and the program in [3]. Pose estimation methods v 01. b 26

Re-projection error https: //en. wikipedia. org/wiki/Reprojection_error • Pose estimation methods v 01. b 27

Epnp references and program • [1] Barycentric coordinate system https: //en. wikipedia. org/wiki/Barycentric_coordinate_system • [2] Vincent Lepetit · Francesc Moreno-Noguer · Pascal Fua, "EPn. P: An Accurate O(n) Solution to the Pn. P Problem", International Journal of Computer Vision , February 2009 web-ink • [3] https: //www. epfl. ch/labs/cvlab/software/multi-viewstereo/epnp/ – https: //www. epfl. ch/labs/cvlab/wpcontent/uploads/2018/08/EPn. P_matlab. zip • [4] Aruco tracker https: //docs. opencv. org/trunk/d 5/dae/tutorial_aruco_detection. ht ml Pose estimation methods v 01. b 28

References • POSIT: POSE estimation – – – • Epnp – • http: //home. in. tum. de/~grembowi/ar 2004_05/3 d. Pose. Estimation_presentation. pdf Vincent Lepetit · Francesc Moreno-Noguer · Pascal Fua, "EPn. P: An Accurate O(n) Solution to the Pn. P Problem", International Journal of Computer Vision , February 2009 https: //pdfs. semanticscholar. org/6 ed 0/083 ff 42 ac 966 a 6 f 37710 e 0 b 5555 b 98 fd 7565. pdf Khwong – • Daniel F. Dementhon, Larry S. Davis, “Model Based Object Pose in 25 Lines of Code ”, http: //home. in. tum. de/~grembowi/ar 2004_05/3 d. Pose. Estimation_presentation. pdf. POSIT, International Journal of Computer Vision June 1995, Volume 15, Issue 1– 2, pp 123– 141 the original paper Sebastian Grembowietz , “ 3 D Pose Estimation” http: //home. in. tum. de/~grembowi/ar 2004_05/3 d. Pose. Estimation_elaboration. pdf a tutorial on pose estimation methods: 3 pt , p 4+p, POSIT, linear method, perspective p 7 p, Thomas Petersen , “A comparison of 2 D-3 D Pose Estimation Methods” Aalborg University 2008 http: //www. haowuhw. com/pose/report. pdf a report on POS, POSIT Daniel Grest, Thomas Petersen, Volker Krüger, “A Comparison of Iterative 2 D-3 D Pose Estimation Methods for Real-Time Applications”, Scandinavian Conference on Image Analysis, SCIA 2009: Image Analysis pp 706 -715 Tutorial ppt • Sebastian Grembowietz, “Algorithms for Augmented Reality 3 D Pose Estimation”, 2004 nov 19, Code – – http: //www. cse. cuhk. edu. hk/%7 Ekhwong/j 2004_IEEE_chang_MM_xlowe_draft. pdf http: //people. rennes. inria. fr/Eric. Marchand/pose-estimation/tutorial-pose-dementhon-opencv. html http: //www. cfar. umd. edu/~daniel/Site_2/Code. html (posit matlab , c++ etc) Pose estimation methods v 01. b 29

appendix Pose estimation methods v 01. b 30

POS, POSIT (alternative formulation) Ref : Thomas Petersen , “A comparison of 2 D-3 D Pose Estimation Methods” Aalborg University 2008 http: //www. haowuhw. com/pose/report. pdf a report on POS, POSIT A simplified model , focal length =f, ox=oy=0 Kint= intrinsic parameters, Kext=extrinsic parameters, projection=Kint*kext 31 • Pose estimation methods v 01. b

POS Algo • Pose estimation methods v 01. b 32