A coordinate location method in point-reader was proposed based on JSEG algorithm in terms of the color diversity of a point reading book. Firstly, the coordinate system of a book is acquired by using adjacent pixels difference and binary in LUV. Secondly, the JSEG algorithm combining texture and color characteristics is applied to the image segmentation of target image. Finally, the area of a point reading pen is extracted based on its minimum color, the writing hand is distinguished based on curvature of a point reading pen’s nib. Then locating the nib coordinate by row scanning. The experiment results show that the method has high accuracy of the reading coordinate, solves the problem of nib coordinate in different holding pen’s ways and decreases the influence of uneven illumination.